ATLAS-OLMo-3-7B-Think-v4/README.md

---
language: en
license: apache-2.0
library_name: transformers
tags:
  - olmo3
  - mirror
  - redistribution
  - bf16
  - thinking-model
  - reasoning
  - atlas
pipeline_tag: text-generation
---

# ATLAS-OLMo-3-7B-Think-v4 — an unmodified **mirror** of `allenai/Olmo-3-7B-Think`

> ⚠️ **Read this first — what this repository is.**
> The weights here are a **byte-identical redistribution of [`allenai/Olmo-3-7B-Think`](https://huggingface.co/allenai/Olmo-3-7B-Think)**, created and trained by the **Allen Institute for AI (Ai2)**.
> **We did not train, fine-tune, quantize, distil, merge, adapt or otherwise modify these weights.**
> This repo exists only so that the [ATLAS](https://github.com/web3guru888/ATLAS) inference engine has a pinned, self-contained copy to serve. **If you want the weights, prefer the original: [`allenai/Olmo-3-7B-Think`](https://huggingface.co/allenai/Olmo-3-7B-Think).**
> Everything measured below measures **the ATLAS serving stack**, not the weights — the weights' quality is entirely Ai2's work.

## Provenance & attribution

**Upstream**: [`allenai/Olmo-3-7B-Think`](https://huggingface.co/allenai/Olmo-3-7B-Think) · Allen Institute for AI · Apache-2.0
**Relationship**: mirror / verbatim redistribution. **Not a derivative work of the weights.**

Verify it yourself — every weight shard in this repo has the same SHA-256 as the corresponding shard in Ai2's repo (checked 2026-08-03 against the Hub's `paths-info` API):

| File | Size (bytes) | SHA-256 | Identical to Ai2? |
|---|---|---|---|
| `model-00001-of-00003.safetensors` | 4,969,984,976 | `611c266f99a1e0575ba9f595e2f44e103aaa03a0c83b244725d6598aee934a7d` | ✅ yes |
| `model-00002-of-00003.safetensors` | 4,981,161,496 | `a032a27f1768184ad0d955752e3564c14b7d1c673d1630f0df7429a439da4f41` | ✅ yes |
| `model-00003-of-00003.safetensors` | 4,644,917,240 | `50ba91f88796c1ed04cacdb9271fa02d213722734dffe05d399eb27bcff7129b` | ✅ yes |
| `config.json` | 1,631 | (git blob `360f3bb9…`) | ✅ yes |

```bash
# reproduce
huggingface-cli download openhubresearch/ATLAS-OLMo-3-7B-Think-v4 model-00001-of-00003.safetensors
huggingface-cli download allenai/Olmo-3-7B-Think            model-00001-of-00003.safetensors
sha256sum ~/.cache/huggingface/hub/models--*/snapshots/*/model-00001-of-00003.safetensors
```

**Honest small print — where this mirror is *not* byte-identical:**

- `tokenizer.json` is **functionally identical but not byte-identical**: same 100,278-entry vocabulary, same 100,000 merges, same added tokens — it was simply re-serialized by a different `tokenizers` version (legacy `"a b"` merge strings vs. Ai2's `["a","b"]` pairs), which is why the file is 3.0 MB here and 7.1 MB upstream. Verified equal after normalisation.
- This mirror **omits** several files Ai2 ships: `merges.txt`, `vocab.json`, `chat_template.jinja`, `olmo-think.png`. If you need those, use the upstream repo.

**Metadata note (2026-08-03).** This repo previously declared `base_model: allenai/Olmo-3-7B-Think` in its card metadata, which caused the Hub to auto-derive the tag **`base_model:finetune:allenai/Olmo-3-7B-Think`** and render *"Finetuned from allenai/Olmo-3-7B-Think"* on this page. **That was wrong — nothing here is fine-tuned.** The Hub's `base_model_relation` field currently accepts only `adapter`, `finetune`, `merge` and `quantized` ([Hub docs](https://huggingface.co/docs/hub/model-cards#specifying-a-base-model)) — there is no value meaning *"verbatim mirror"* — so the `base_model` field has been **removed** rather than left carrying a claim we cannot support, and attribution is stated explicitly in prose instead. If you are looking for something we actually trained, it is the LoRA adapter at [`openhubresearch/ATLAS-Taurus-Expert-7B-poc-v1`](https://huggingface.co/openhubresearch/ATLAS-Taurus-Expert-7B-poc-v1) (a proof-of-concept: +0.021 keyword-overlap on 12 held-out questions — a small, noisy result, honestly reported there).

## Licence & credit

These weights are © Allen Institute for AI, released under the **Apache License 2.0**, and are redistributed here **unmodified** under that licence. Ai2's model card, licence terms and intended-use guidance for the original are authoritative: <https://huggingface.co/allenai/Olmo-3-7B-Think>. For errors in Ai2's model card, Ai2 asks that you contact `olmo@allenai.org` — **please do not contact Ai2 about this mirror**; open a discussion here instead.

```bibtex
@misc{olmo2025olmo3,
  title  = {Olmo 3},
  author = {Team Olmo and others},
  year   = {2025},
  eprint = {2512.13961},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url    = {https://arxiv.org/abs/2512.13961}
}
```

---

## What ATLAS adds (the serving stack — *not* the weights)

**Production endpoint — what it actually serves today.** ATLAS runs an OpenAI-compatible API at `https://atlas.thebeastagi.com/v1` (bearer-key auth; `GET /v1/models` and `GET /health` open). ⚠️ **As of 2026-08-03 that endpoint serves only `olmo3-32b` — it does *not* currently serve this 7B model.** `GET /v1/models` returns exactly one entry, `olmo3-32b`; check it yourself before assuming otherwise. An earlier version of this card said the endpoint served "this model", which was true when the 7B was deployed and became false after the 32B cutover. To run the 7B, build ATLAS and serve these weights locally — see [Quick Start with ATLAS](#quick-start-with-atlas) below. An [OpenRouter](https://openrouter.ai) provider application is in progress.

## Latest Release — v4.2.0 (2026-07-06): Reference Fidelity

**[v4.2.0 — HF-Reference Fidelity + Reasoning API](https://github.com/web3guru888/ATLAS/releases/tag/v4.2.0)** · 627 tests · CI green

Differential testing against HuggingFace `transformers` (token-level logit comparison) uncovered and fixed **three silent inference-quality bugs** that had been degrading output since the initial port:

1. **YaRN correction range** — missing 2π factor in the wavelength calculation, incorrect interpolation ramp, and `attn_factor` must be applied **squared**
2. **Layer-type RoPE split** — YaRN scaling applies only to the 8 full-attention layers; the 24 sliding-window layers use plain θ=500,000 (per `configuration_olmo3.py`)
3. **QK-norm scope** — RMSNorm is computed over the **full 4096-dim** Q/K projection, *not* per-head — the per-head variant silently destroyed retrieval beyond ~128 tokens

ATLAS output is now **byte-identical to the HF reference implementation**. To our knowledge, ATLAS is the only from-scratch (non-`transformers`, zero-dependency Rust) OLMo-3-7B-Think implementation serving at full reference quality.

**Serving upgrades in v4.2.0:**
- Context window raised to **16,384 tokens** served (model supports up to 64K via YaRN), max output 3,584 tokens
- Official OLMo chat template with server-side `<think>` primer and the Ai2 reference system prompt
- **Reasoning API**: chain-of-thought surfaced as `message.reasoning` and streaming `delta.reasoning` (OpenRouter convention) — final answers stay clean, reasoning stays inspectable
- Sampling defaults aligned with the Ai2 reference (`repetition_penalty=1.0` — penalties make thinking models ramble)
- `compute-sanitizer` memcheck clean; exact CPU/GPU parity verified on long sequences

### Benchmarks — these measure the ATLAS **serving stack** running Ai2's unmodified weights

These numbers are *not* a claim about model quality: they show ATLAS's own inference bugs being fixed until it reproduced Ai2's reference behaviour. Credit for the model's capability belongs to Ai2.

(measured on the live A100 endpoint, before → after the ATLAS fidelity fixes)

| Benchmark | Before fixes | **After fixes (v4.2.0)** | Reference |
|-----------|-------------|--------------------------|-----------|
| GSM8K (25 problems) | 12% | **88%** | Ai2 reported: 85–91% ✅ |
| MMLU (100 questions, direct) | 22% | **54%** | — |
| HumanEval (15 tasks, execution-verified) | — | **73.3% pass@1** | — |
| Needle-in-a-haystack | failed >128 tok | **clean at 3.3K+** | — |

All remaining misses were output-length truncations; accuracy was 100% on items where reasoning completed within budget.

<details>
<summary>Previous release — v4.1.0 (Full GPU Attention + StigmergicHook)</summary>

- **4× throughput**: 61.7 tok/s BF16 on A100 (up from 15.4 tok/s) — zero intra-layer PCIe transfers during decode
- **Full GPU attention path**: custom CUDA kernels handle the entire decode step on-device
- **New CUDA kernels**: `decode_attention_kernel`, `qk_norm_inplace_kernel`, `rope_precomputed_kernel`, `kv_cache_write_kernel`, `atlas_gpu_argmax`
- **New crate `atlas-infer`**: exposes the `StigmergicHook` trait — a GraphPalace bridge that lets inference hooks read/write stigmergic memory in real time during token generation

</details>

## Model Details

| Property | Value |
|----------|-------|
| **Base model** | [allenai/Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think) |
| **Architecture** | Olmo3ForCausalLM — post-norm + QK-norm, SWA + YaRN RoPE |
| **Parameters** | 7.3B |
| **Precision** | BF16 (bfloat16) |
| **Context** | 65,536 tokens max (YaRN factor=8); served at 16,384 in v4.2.0 |
| **Vocab** | 100,278 tokens |
| **License** | Apache 2.0 |
| **Inference engine** | ATLAS v4.2.0 — pure Rust, zero external crate dependencies |
| **Fidelity** | Byte-identical logits vs HF `transformers` reference |
| **ATLAS tests** | 627 passing, CI green |

> **Note on model weights**: see [Provenance & attribution](#provenance--attribution) above — these are Ai2's unmodified weights; ATLAS is the inference + memory-palace engine, not a fine-tune.

## ATLAS Inference Engine

This model is verified to run correctly with [ATLAS](https://github.com/web3guru888/ATLAS), a pure-Rust LLM inference framework with zero external crate dependencies. ATLAS implements the full OLMo-2/3 architecture from scratch:

- **Post-norm layer ordering** — `x = residual + rmsnorm(output)` matching the [HuggingFace Olmo2DecoderLayer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/olmo2/modeling_olmo2.py) reference
- **QK-norm** — RMSNorm over the full 4096-dim Q/K projections before RoPE (v4.2.0 fix)
- **Sliding Window Attention** — 24/32 layers with window=4,096, plain θ=500K RoPE; YaRN on the 8 full-attention layers only (v4.2.0 fix)
- **YaRN RoPE** — factor=8, original_max_seq_len=8,192, corrected 2π wavelength range, attn_factor² (v4.2.0 fix)
- **BF16 W16A32** — weights in BF16 (14 GB VRAM), activations in f32
- **Official OLMo chat template** — with `<think>` primer and Ai2 system prompt (v4.2.0)
- **Reasoning API** — `message.reasoning` / streaming `delta.reasoning` (v4.2.0)
- **Full sampling pipeline** — repetition penalty, temperature, top-p, top-k, min-p, frequency/presence penalty
- **Full GPU attention path** (v4.1.0) — five custom CUDA kernels, zero PCIe transfers during decode
- **`StigmergicHook` trait** (v4.1.0, `atlas-infer` crate) — GraphPalace bridge for live stigmergic memory reads/writes during token generation

### Performance (A100-SXM4-40GB, production serving)

| Metric | Value |
|--------|-------|
| Decode throughput | ~50 tok/s sustained (BF16, live endpoint); 61.7 tok/s peak micro-bench |
| TTFT | ~1.5 s (short prompts) |
| VRAM | ~14 GB |
| CPU/GPU parity | exact match (long-sequence test) |
| Model load time | ~108 s (3 shards, 14 GB) |

Known gaps being tracked for the next cycle: [batched prefill](https://github.com/web3guru888/ATLAS/issues/22), [GEMV occupancy](https://github.com/web3guru888/ATLAS/issues/23), [32K context via BF16 KV cache](https://github.com/web3guru888/ATLAS/issues/24).

## Quick Start with ATLAS

```bash
# Build ATLAS from source
git clone https://github.com/web3guru888/ATLAS.git
cd ATLAS
cargo build --release -p atlas-cli

# Download model weights
# (or use huggingface-cli: hf download openhubresearch/ATLAS-OLMo-3-7B-Think-v4)

# Start OpenAI-compatible API server
./target/release/atlas api serve \
  --weights /path/to/ATLAS-OLMo-3-7B-Think-v4 \
  --model olmo3-7b \
  --port 8080

# Query the API (reasoning is returned in message.reasoning)
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "olmo3-7b",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "max_tokens": 1500,
    "temperature": 0.6,
    "top_p": 0.95
  }'
```

> **Tip**: OLMo-3-Think reasons before answering — give it `max_tokens ≥ 1500` so the chain of thought can complete. Leave `repetition_penalty` at 1.0 (Ai2 reference).

## Quick Start with HuggingFace Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "openhubresearch/ATLAS-OLMo-3-7B-Think-v4",
    torch_dtype="bfloat16",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
    "openhubresearch/ATLAS-OLMo-3-7B-Think-v4"
)

messages = [{"role": "user", "content": "What is the capital of France?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True,
    return_tensors="pt", return_dict=True
).to(model.device)

output = model.generate(
    **inputs, max_new_tokens=1500,
    temperature=0.6, top_p=0.95, do_sample=True,
)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## What's Next: the 32B chapter

With the 7B substrate at reference fidelity, work is beginning on serving **OLMo-3-32B-Think** on the same A100-40GB via 4-bit weight quantization — tracked in [web3guru888/ATLAS#26](https://github.com/web3guru888/ATLAS/issues/26).

## About ATLAS

ATLAS (**A**ctive-inference **T**raining with **L**earned **A**daptive **S**tigmergy) is a next-generation LLM framework built in pure Rust with zero external crate dependencies — the SQLite principle applied to AI infrastructure. It fuses:

- **GraphPalace** — Stigmergic memory palace with pheromone-guided navigation
- **ASTRA** — Live discovery engine hitting NASA, WHO, World Bank APIs
- **TRM-CausalValidator** — 7M-param recursive validator
- **Champagnat n-Morphic Framework** — biologically-grounded training dynamics

22 crates. 627 tests. One coherent system. Zero external Rust dependencies.

**Website**: [atlasagi.org](https://atlasagi.org) · **Observatory**: [Interactive Demo](https://huggingface.co/spaces/openhubresearch/ATLAS) · **Live API** (32B only, not this model): [atlas.thebeastagi.com](https://atlas.thebeastagi.com/v1/models) · **Organization**: [OpenHub Research](https://openhubresearch.org/) · **Author**: Robin Dey

## Citing

**If you use the model, cite Ai2's Olmo 3** (see [Licence & credit](#licence--credit) above) — not this repo. The BibTeX below is for the **ATLAS inference engine only**.

```bibtex
@software{atlas2026,
  title       = {ATLAS: Active-inference Training with Learned Adaptive Stigmergy},
  author      = {Robin Dey},
  year        = {2026},
  institution = {OpenHub Research, Thailand},
  url         = {https://github.com/web3guru888/ATLAS},
  note        = {Pure Rust LLM framework. v4.2.0: 22 crates, 627 tests,
                 OLMo-3-7B-Think at HF-reference fidelity on A100 (BF16,
                 full GPU attention, reasoning API). GSM8K 88%, MMLU 54%,
                 HumanEval 73.3% pass@1 through the ATLAS serving stack.}
}
```