coneml-810m-alpha-arithmetic/README.md

---
license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
---

# ConeML 810M Alpha-Arithmetic

ConeML 810M Alpha-Arithmetic is an 810,112,512-parameter, decoder-only
language model trained from scratch and instruction-tuned for a narrow
small-number arithmetic profile. It shares its pretrained base with
`coneml-810m-alpha` and differs in fine-tuning composition.

This variant records the stronger carry/borrow profile of the pair. The
companion `coneml-810m-alpha` has substantially stronger function-writing
and two-step results. Results are specific to the documented prompt
formats and task families, not claims of general mathematical, coding,
reasoning, or factual-knowledge performance.

- Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied
  embeddings, 8192 context, 32,768-token BPE tokenizer
- Pretraining corpus: 27.87B curated tokens
- Selected pretrained-base exposure: approximately 12.32B token
  positions, or 15.2 tokens per parameter
- Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090;
  fine-tuning and evaluation are excluded
- Release `model.safetensors` SHA-256:
  `e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1`

## 1. Measured profile

Across four held-out core lane sets—1-digit addition, 1-digit
multiplication, carry addition, and borrow subtraction—the model scores
1,093/1,116 (97.9%). Two of those four held-out sets scored 100%.

This is a result on four explicitly named, task-family-matched sets. It is
not a claim of 97.9% general arithmetic accuracy. ConeML internal
certification is not independent third-party certification.

| held-out task family | result | n |
|---|---:|---:|
| 2–3-digit addition with carry | 96.8% | 500 |
| 2-digit subtraction with borrow | 98.6% | 500 |
| 1-digit addition, including 19 zero-operand items | 100% | 71 |
| 1-digit multiplication | 100% | 45 |
| "Which is bigger, X or Y?" | 93.7% | 300 |
| comparison word problems | 79.0% | 500 |
| missing addend | 67.4% | 500 |
| two-step word problems | 43.8% | 500 |
| 2-digit × 1-digit multiplication | 83.6% | 500 |
| missing factor | 18.0% | 500 |
| Python function writing (single function, executed) | 33.7% | 300 |

Behavioral screens:

- Designated-refusal probe: 17/17 refusal prompts produced refusals, and
  4/5 in-scope contrast prompts were answered correctly. The miss was an
  incorrect comparison answer, not an over-refusal. This is a small
  targeted policy test, not general epistemic calibration.
- Conversation-v2: 8/8 turns passed in one scripted dialogue covering
  cross-turn arithmetic, story revision, comparison explanation, and one
  unknown-personal-fact refusal. This is a single dialogue, not a general
  conversation benchmark.
- Everyday-reasoning screen: 13/20 by manual adjudication; the
  phrase-sensitive automatic scorer recorded 6/20.

## 2. Peer comparison and efficiency

The pair was tested against five post-trained peers on the same reduced
diagnostic screen, through each model's native instruction interface,
with greedy decoding, repetition penalty 1.15, and matched short-answer
budgets.

| model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | transitive names d1/d3/d5 (n=32 each) | designated refusals (n=17) |
|---|---:|---:|---:|---:|---:|
| ConeML 810M Alpha | 0.81B | 421 (72.0%) | 83 | 23/15/15 | **13** |
| ConeML 810M Alpha-Arithmetic | 0.81B | **442 (75.6%)** | 35 | 26/17/16 | 11 |
| Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 23/22/10 | 0 |
| Qwen3 | 0.6B | 223 (38.1%) | **98** | 18/10/9 | 1 |
| Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 16/7/5 | 1 |
| TinyLlama Chat | 1.1B | 113 (19.3%) | 51 | 8/18/22 | 0 |
| SmolLM2 Instruct | 1.7B | 380 (65.0%) | 96 | 16/6/5 | 1 |

This is not a general leaderboard: the task families match ConeML's
trained surfaces. On this screen, the ConeML pair led mixed arithmetic;
ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing
but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero
over-refusals on five in-scope contrasts.

On the question-form name-chain depth screen, ConeML 810M
Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5;
ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic
variant exceeded Qwen3 and Llama 3.2 at all three shown depths and
Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama
led the group at depth 5. Entity-chain controls were mixed and are
reported in the full table. Chance is 1/(depth+1); this is an
exact-selection surface test, not a claim of general reasoning.

Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with
this release's 442/585 (75.6%), while using at least 17.6 times the
generated-token budget per item and approximately 24 times the recorded
wall time. Runtime ratios are implementation-specific.

ConeML's selected base consumed 12.32B token positions—15.2 tokens per
parameter and approximately 5.99e19 training FLOPs under the `6ND`
convention. Pretraining took approximately 11 days on one RTX 5090.
Assuming, rather than claiming to have metered, a 0.70 kW average wall
draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately
CHF 26–55 of marginal pretraining electricity. This excludes hardware,
labor, SFT, evaluation, conversion, and emissions accounting.

The complete category table, peer wins, adverse base result, evaluated
revisions, assumptions, and frozen evidence commitments are in
[`eval/PEER_COMPARISON.md`](eval/PEER_COMPARISON.md).

## 3. Standard benchmark limits

Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses
strict final-number matching; HumanEval reports pass@1 from executed
tests.

| benchmark | this model | same base, no fine-tune |
|---|---:|---:|
| GSM8K (test, first 200) | 10/200 (5.0%) | 7/200 (3.5%) |
| HumanEval (pass@1, tests executed) | 3/164 (1.8%) | 0/164 (0.0%) |

These results are weak and define the boundary of the specialization:
the four-lane arithmetic profile does not transfer to broad GSM8K or
HumanEval performance.

## 4. Supported prompt formats

Evaluated decoding is greedy with repetition penalty 1.15, as pinned in
`generation_config.json`.

```text
# task format (single turn)
Question: <task>
Answer:

# dialogue format (chat_template.jinja)
User:
<message>
Assistant:
```

Dialogue generation ends at `<|endoftext|>`. Prompting outside these
formats, including paraphrases, was not systematically evaluated except
for the published raw-completion spot check.

## 5. Quantized variants

Q8_0 GGUF was evaluated separately under llama.cpp. Its runtime and stop
handling differ from the bf16 harness, so score differences cannot be
attributed solely to quantization.

| held-out task family | Q8_0 |
|---|---:|
| subtraction with borrow | 98.6% (n=500) |
| addition with carry | 92.6% (n=500) |
| 1-digit multiplication | 100% (n=45) |
| two-step word problems | 32.6% (n=500) |

For local GGUF inference:

```bash
llama-cli \
  -m coneml-810m-alpha-arithmetic-Q8_0.gguf \
  -p $'Question: What is 47 + 38?\nAnswer:' \
  -n 32 --repeat-penalty 1.15 --no-conversation --single-turn
```

For Ollama, download `Modelfile` beside the Q8_0 file, then run:

```bash
ollama create coneml-810m-alpha-arithmetic -f Modelfile
ollama run coneml-810m-alpha-arithmetic "What is 47 + 38?"
```

## 6. Loading with Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "ConeML/coneml-810m-alpha-arithmetic"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Question: What is 47 + 38?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=False,
    repetition_penalty=1.15,
)
answer = tokenizer.decode(
    output[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)
```

This is a text-only causal language model. Load it with
`AutoModelForCausalLM` or `LlamaForCausalLM`, not a multimodal model class.

## 7. Intended use and safety

This release is intended for research and constrained local workflows
inside its measured small-number arithmetic envelope. It is not a general
mathematics system. Validate outputs whenever exactness matters.

It is not intended for factual retrieval, high-stakes medical, legal, or
financial decisions, autonomous action, or unsandboxed execution of
generated code. Generated code may be incomplete or unsafe; inspect and
test it in an isolated environment.

## 8. Known limitations

- The measured strength is narrow. Missing factor scored 18.0%, two-step
  word problems 43.8%, 2-digit × 1-digit multiplication 83.6%, and
  executed single-function writing 33.7%.
- Not a code model: complete programs scored 0/10, executable SQL 1/10,
  and bash 1/5.
- The model was not evaluated as a factual-QA system and refused all six
  real-world factual prompts in the designated-refusal probe.
- Money/decimal change-making scored 0/3 on the held-out screen; the model
  also showed yes-bias on feasibility questions.
- Multi-step GSM-class reasoning remained weak.
- Frameless prompting was spot-checked, not systematically certified.

## 9. Evidence and reproducibility

The `eval/` directory publishes:

- `summary.json`: all aggregate bf16 results used above;
- `representative-samples.json`: deterministic, rule-selected examples
  from ConeML-owned probes;
- `EVALUATION_METHODOLOGY.md`: evaluation definitions and scope;
- separate Q8_0 and focused aggregate results; and
- `PRIVATE_EVIDENCE_SHA256SUMS.txt`: commitments to the complete
  row-level evidence retained by ConeML.
- `peer-comparison-summary.json`, `PEER_COMPARISON.md`, and
  `PEER_EVIDENCE_SHA256SUMS.txt`: the peer results, interpretation, and
  commitments to the retained peer-generation evidence.

Complete proprietary probe rows and generations are retained privately to
avoid releasing the full measurement instruments. The hashes permit later
disclosure to be checked against the evidence frozen at release time.
Public GSM8K and HumanEval rows are not redistributed; only aggregate
results are published. `SHA256SUMS.txt` covers every shipped file.

## 10. Research license and commercial work

Released for non-commercial use under CC BY-NC 4.0. Commercial use is not
granted by this release. See `LICENSE.md`.

For commercial evaluation, model-engineering, or licensing enquiries,
contact [contact@coneml.com](mailto:contact@coneml.com) or visit
[ConeML for Organizations](https://coneml.com/organizations/). Commercial availability does not
imply that this alpha checkpoint is suitable for an unvalidated
production workflow.