d24-sft-v5-nogsm8k/README.md

---
license: other
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model: ftajwar/d24-climbmix-dolmino-midtrain-100b
datasets:
- sfanm/d24-sft-mixture
tags:
- llama
- causal-lm
- chat
- sft
- rlvr
- gsm8k-free-sft
- contamination-control
- gpt2-tokenizer
---

# d24-sft-v5-nogsm8k

`d24-sft-v5-nogsm8k` is an experimental **756,819,456-parameter**
English chat model intended as a clean SFT-exposure control for GSM8K
reinforcement-learning experiments. Its supervised fine-tuning mixture omits
the complete 7,473-example GSM8K training component.

All 756,819,456 parameters in the published safetensors file are BF16.

> **Scope of the no-GSM8K claim:** the explicit `openai/gsm8k` component was
> removed from supervised fine-tuning. This is not a claim that the much larger
> pretraining and midtraining corpora contain no GSM8K overlap, paraphrases, or
> similar grade-school mathematics. Other non-GSM8K SFT sources can also contain
> mathematical material.

The standard-mixture sibling, which did include GSM8K during SFT, is
[`sfanm/d24-sft-v5-simplechat`](https://huggingface.co/sfanm/d24-sft-v5-simplechat).

## Lineage

1. [`ftajwar/d24-climbmix-100b`](https://huggingface.co/ftajwar/d24-climbmix-100b):
   pretrained from scratch on about 100B tokens of ClimbMix.
2. [`ftajwar/d24-climbmix-dolmino-midtrain-100b`](https://huggingface.co/ftajwar/d24-climbmix-dolmino-midtrain-100b):
   continued-pretrained for another approximately 100B tokens on 80% OLMo-3
   Dolmino plus 20% ClimbMix replay.
3. This checkpoint: one epoch of SFT on the d24 conversation mixture after
   removing the GSM8K component, formatted with the simple ChatML template.

The model has seen approximately 200B pretraining/midtraining tokens before SFT.
The `v5` label identifies this training lineage; it is not a parameter-count
label.

## What was removed

The no-GSM8K corpus was derived from the standard
[`sfanm/d24-sft-mixture`](https://huggingface.co/datasets/sfanm/d24-sft-mixture)
source mixture. The standard and filtered simple-ChatML preparation summaries
differ by exactly 7,473 source conversations, the size of the GSM8K training
split used by the standard mixture:

| Preparation count | Standard mixture | No-GSM8K mixture | Difference |
|---|---:|---:|---:|
| Training source conversations | 821,884 | 814,644 | 7,240 |
| Validation source conversations | 26,514 | 26,281 | 233 |
| Total source conversations | 848,398 | 840,925 | **7,473** |
| Packed training sequences | 227,443 | 226,879 | 564 |
| Packed validation sequences | 7,381 | 7,363 | 18 |
| Packed tokens | 476,320,096 | 475,114,114 | 1,205,982 |

Both preparations used sequence length 2,048, `first_fit_shuffle`, seed 42,
the GPT-2 tokenizer, and the same simple ChatML template. Removing source
conversations changes packing boundaries, so packed-sequence differences are
not expected to equal the source-conversation difference.

## Architecture

| Field | Value |
|---|---:|
| Parameters | 756,819,456 |
| Layers | 24 |
| Hidden size | 1,536 |
| Attention heads | 12 (MHA) |
| FFN size | 4,096 (SwiGLU/SiLU) |
| Position encoding | RoPE, theta 10,000 |
| Normalization | RMSNorm |
| Embeddings | Tied |
| Tokenizer | GPT-2 BPE, vocabulary padded to 50,304 |
| Maximum context | 2,048 tokens |
| Published weight dtype | BF16 |

The Hugging Face architecture is `LlamaForCausalLM`, but the model uses the
GPT-2 tokenizer and the d24 architecture above. It is not a Llama-family
pretrained checkpoint.

## SFT training

| Hyperparameter | Value |
|---|---:|
| Epochs / steps | 1 / 1,773 |
| Global / micro batch | 128 / 1 |
| Sequence length | 2,048 |
| Optimizer | AdamW |
| Peak / minimum LR | 1e-4 / 1e-5 |
| LR schedule | Cosine, 50-step warmup |
| Adam betas / epsilon | (0.9, 0.95) / 1e-8 |
| Weight decay | 0.1 |
| Gradient clip | 1.0 |
| Precision | BF16 |

The final SFT validation language-model loss was **0.797625** (perplexity
**2.220262**). This is an in-distribution SFT validation metric, not a
downstream capability or safety evaluation.

## Chat format and stopping

The template is:

```text
<|im_start|>user
...<|im_end|>
<|im_start|>assistant
...
```

`<|im_end|>` is a literal string represented by ordinary GPT-2 BPE tokens. It
is **not** `eos_token_id=50256` and is not a registered special token.
Generation must stop on the string `<|im_end|>`; otherwise the model can
continue into another turn until `max_new_tokens`.

## Transformers usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sfanm/d24-sft-v5-nogsm8k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": "Natalia sold clips to 48 friends in April and half as many in May. How many total?",
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    stop_strings=["<|im_end|>"],
    tokenizer=tokenizer,
)
response = tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=False,
)
print(response)
```

For vLLM, use `SamplingParams(stop=["<|im_end|>"])`.

## Intended use

- controlled studies of learning GSM8K through RLVR without explicit GSM8K
  examples during SFT;
- comparisons against the GSM8K-exposed v5 simplechat sibling;
- research on GRPO, RLOO, maxRL, SFT, and small-model behavior.

## Limitations

- The no-GSM8K designation applies only to the explicit SFT mixture component;
  it does not prove global benchmark decontamination across pretraining and
  midtraining.
- This is a small experimental research model and has not undergone production
  safety alignment or a comprehensive capability/safety evaluation.
- It can produce incorrect, incoherent, biased, or unsafe text. Verify outputs
  independently and do not use it for high-stakes decisions.
- The context window is 2,048 tokens.
- Chat generation is incorrect unless the caller stops on the literal
  `<|im_end|>` string.
- The model and SFT mixture derive from multiple upstream datasets. The Hub
  metadata therefore uses `license: other`; review source licenses and terms
  before redistribution or downstream use.

## Reproducibility

- SFT checkpoint: `sft-d24-v5-nogsm8k/iter_0001773`
- HF export: `d24-sft-v5-nogsm8k-hf`
- Packed corpus: `sft-pack-d24-nogsm8k`
- Chat template: `src/nemotron/data_prep/templates/simple_chatml.jinja`
- Training repository: `yudasong/unified-training`