coneml-810m-alpha/README.md

---
license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
---

# ConeML 810M Alpha

ConeML 810M Alpha is an 810,112,512-parameter, decoder-only language
model trained from scratch and instruction-tuned for constrained English
tasks. Its measured profile favors single Python functions, selected
small-number arithmetic families, two-step word problems, and a narrow
designated-refusal policy.

Results are specific to the documented prompt formats and evaluation task
families. They are not claims of general coding, reasoning, factual
knowledge, or production reliability.

- Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied
  embeddings, 8192 context, 32,768-token BPE tokenizer
- Pretraining corpus: 27.87B curated tokens
- Selected pretrained-base exposure: approximately 12.32B token
  positions, or 15.2 tokens per parameter
- Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090;
  fine-tuning and evaluation are excluded
- Release `model.safetensors` SHA-256:
  `e5df8f2edd773ad079ba00d383ace81b96cbb9579d64c570db505f9efc032b76`
- Companion release: `coneml-810m-alpha-arithmetic`, from the same base,
  has substantially higher carry/borrow accuracy and lower function and
  two-step accuracy

## 1. Measured profile

ConeML internal certification uses held-out instances from the same task
families as the training generators. It is not independent third-party
certification. Code is scored by executing each generated function
against held-out tests.

| held-out task family | result | n |
|---|---:|---:|
| Python function writing (single function, executed) | 69.0% | 300 |
| two-step word problems | 79.0% | 500 |
| 2-digit × 1-digit multiplication | 97.4% | 500 |
| 1-digit multiplication | 97.8% | 45 |
| 1-digit addition | 98.6% | 71 |
| "Which is bigger, X or Y?" | 95.0% | 300 |
| 2-digit subtraction with borrow | 91.0% | 500 |
| comparison word problems | 72.6% | 500 |
| missing addend | 71.6% | 500 |
| 2–3-digit addition with carry | 53.2% | 500 |
| missing factor | 17.8% | 500 |

Behavioral screens:

- Designated-refusal probe: 17/17 refusal prompts produced refusals, and
  5/5 in-scope contrast prompts in matching wording were answered. This
  small targeted probe measures adherence to that policy, not general
  epistemic calibration.
- Conversation-v2: 8/8 turns passed in one scripted dialogue covering
  cross-turn arithmetic, story revision, comparison explanation, and one
  unknown-personal-fact refusal. This is a single dialogue, not a general
  conversation benchmark.
- Everyday-reasoning screen: 11/20 by manual adjudication; the
  phrase-sensitive automatic scorer recorded 6/20.

## 2. Peer comparison and efficiency

The pair was tested against five post-trained peers on the same reduced
diagnostic screen, through each model's native instruction interface,
with greedy decoding, repetition penalty 1.15, and matched short-answer
budgets.

| model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | transitive names d1/d3/d5 (n=32 each) | designated refusals (n=17) |
|---|---:|---:|---:|---:|---:|
| ConeML 810M Alpha | 0.81B | 421 (72.0%) | 83 | 23/15/15 | **13** |
| ConeML 810M Alpha-Arithmetic | 0.81B | **442 (75.6%)** | 35 | 26/17/16 | 11 |
| Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 23/22/10 | 0 |
| Qwen3 | 0.6B | 223 (38.1%) | **98** | 18/10/9 | 1 |
| Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 16/7/5 | 1 |
| TinyLlama Chat | 1.1B | 113 (19.3%) | 51 | 8/18/22 | 0 |
| SmolLM2 Instruct | 1.7B | 380 (65.0%) | 96 | 16/6/5 | 1 |

This is not a general leaderboard: the task families match ConeML's
trained surfaces. On this screen, the ConeML pair led mixed arithmetic;
ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing
but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero
over-refusals on five in-scope contrasts.

On the question-form name-chain depth screen, ConeML 810M
Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5;
ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic
variant exceeded Qwen3 and Llama 3.2 at all three shown depths and
Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama
led the group at depth 5. Entity-chain controls were mixed and are
reported in the full table. Chance is 1/(depth+1); this is an
exact-selection surface test, not a claim of general reasoning.

Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with
ConeML 810M Alpha-Arithmetic's 442/585 (75.6%), while using at least 17.6 times the
generated-token budget per item and approximately 24 times the recorded
wall time. Runtime ratios are implementation-specific.

ConeML's selected base consumed 12.32B token positions—15.2 tokens per
parameter and approximately 5.99e19 training FLOPs under the `6ND`
convention. Pretraining took approximately 11 days on one RTX 5090.
Assuming, rather than claiming to have metered, a 0.70 kW average wall
draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately
CHF 26–55 of marginal pretraining electricity. This excludes hardware,
labor, SFT, evaluation, conversion, and emissions accounting.

The complete category table, peer wins, adverse base result, evaluated
revisions, assumptions, and frozen evidence commitments are in
[`eval/PEER_COMPARISON.md`](eval/PEER_COMPARISON.md).

## 3. Standard benchmark limits

Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses
strict final-number matching; HumanEval reports pass@1 from executed
tests.

| benchmark | this model | same base, no fine-tune |
|---|---:|---:|
| GSM8K (test, first 200) | 12/200 (6.0%) | 7/200 (3.5%) |
| HumanEval (pass@1, tests executed) | 5/164 (3.0%) | 0/164 (0.0%) |

These results are weak and define an important boundary: the narrower
task-family results above do not transfer to broad GSM8K or HumanEval
performance.

## 4. Supported prompt formats

Evaluated decoding is greedy with repetition penalty 1.15, as pinned in
`generation_config.json`.

```text
# task format (single turn)
Question: <task>
Answer:

# dialogue format (chat_template.jinja)
User:
<message>
Assistant:
```

Dialogue generation ends at `<|endoftext|>`. Prompting outside these
formats, including paraphrases, was not systematically evaluated except
for the published raw-completion spot check.

## 5. Quantized variants

Q8_0 GGUF was evaluated separately under llama.cpp. That runtime and its
stop handling differ from the bf16 harness, so the scores are not directly
comparable and differences cannot be attributed solely to quantization.

| held-out task family | Q8_0 |
|---|---:|
| Python function writing | 75.0% (n=300) |
| two-step word problems | 57.0% (n=500) |
| addition with carry | 40.2% (n=500) |
| multiplication families | 94.8–95.6% |
| which-bigger | 92.7% (n=300) |

Use bf16 when the bf16 profile is required.

For local GGUF inference:

```bash
llama-cli \
  -m coneml-810m-alpha-Q8_0.gguf \
  -p $'Question: What is 7 + 8?\nAnswer:' \
  -n 32 --repeat-penalty 1.15 --no-conversation --single-turn
```

For Ollama, download `Modelfile` beside the Q8_0 file, then run:

```bash
ollama create coneml-810m-alpha -f Modelfile
ollama run coneml-810m-alpha "What is 7 + 8?"
```

## 6. Loading with Transformers

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "ConeML/coneml-810m-alpha"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Question: What is 7 + 8?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=False,
    repetition_penalty=1.15,
)
answer = tokenizer.decode(
    output[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)
print(answer)
```

This is a text-only causal language model. Load it with
`AutoModelForCausalLM` or `LlamaForCausalLM`, not a multimodal model class.

## 7. Intended use and safety

This release is intended for research, constrained local inference,
evaluation-harness development, and experiments whose prompts match the
documented formats. Review outputs before use.

It is not intended for factual retrieval, high-stakes medical, legal, or
financial decisions, autonomous action, or unsandboxed execution of
generated code. Generated code may be incomplete or unsafe; inspect and
test it in an isolated environment.

## 8. Known limitations

- Not a general code model: complete programs scored 0/10, executable SQL
  1/10, and bash 1/5. The 69% result is for single functions on ConeML's
  task-family-matched battery.
- Addition with carry is this release's weakest internally certified
  arithmetic family at 53.2%; the arithmetic companion scored 96.8%.
- The model was not evaluated as a factual-QA system and refused all six
  real-world factual prompts in the designated-refusal probe.
- Money/decimal change-making scored 0/3 on the held-out screen; the model
  also showed yes-bias on feasibility questions.
- Multi-step GSM-class reasoning remained weak.
- Frameless prompting was spot-checked, not systematically certified.

## 9. Evidence and reproducibility

The `eval/` directory publishes:

- `summary.json`: all aggregate bf16 results used above;
- `representative-samples.json`: deterministic, rule-selected examples
  from ConeML-owned probes;
- `EVALUATION_METHODOLOGY.md`: evaluation definitions and scope;
- separate Q8_0 aggregate results; and
- `PRIVATE_EVIDENCE_SHA256SUMS.txt`: commitments to the complete
  row-level evidence retained by ConeML.
- `peer-comparison-summary.json`, `PEER_COMPARISON.md`, and
  `PEER_EVIDENCE_SHA256SUMS.txt`: the peer results, interpretation, and
  commitments to the retained peer-generation evidence.

Complete proprietary probe rows and generations are retained privately to
avoid releasing the full measurement instruments. The hashes permit later
disclosure to be checked against the evidence frozen at release time.
Public GSM8K and HumanEval rows are not redistributed; only aggregate
results are published. `SHA256SUMS.txt` covers every shipped file.

## 10. Research license and commercial work

Released for non-commercial use under CC BY-NC 4.0. Commercial use is not
granted by this release. See `LICENSE.md`.

For commercial evaluation, model-engineering, or licensing enquiries,
contact [contact@coneml.com](mailto:contact@coneml.com) or visit
[ConeML for Organizations](https://coneml.com/organizations/). Commercial availability does not
imply that this alpha checkpoint is suitable for an unvalidated
production workflow.