d24-sft-v5-simplechat/README.md

---
license: other
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model: ftajwar/d24-climbmix-dolmino-midtrain-100b
datasets:
- sfanm/d24-sft-mixture
tags:
- llama
- causal-lm
- chat
- sft
- nanochat
- nemotron
- gpt2-tokenizer
---

# d24-sft-v5-simplechat

`d24-sft-v5-simplechat` is an experimental **756,819,456-parameter** English
chat model for research on supervised fine-tuning and reinforcement learning.
It is the canonical simple-ChatML SFT policy from the d24 v5 lineage.

All 756,819,456 parameters in the published safetensors file are BF16.

## Lineage

1. [`ftajwar/d24-climbmix-100b`](https://huggingface.co/ftajwar/d24-climbmix-100b):
   pretrained from scratch on about 100B tokens of ClimbMix.
2. [`ftajwar/d24-climbmix-dolmino-midtrain-100b`](https://huggingface.co/ftajwar/d24-climbmix-dolmino-midtrain-100b):
   continued-pretrained for another approximately 100B tokens on 80% OLMo-3
   Dolmino plus 20% ClimbMix replay.
3. This checkpoint: one epoch of SFT on the standard d24 conversation mixture,
   formatted with the simple ChatML template.

The model has seen approximately 200B pretraining/midtraining tokens before SFT.
The `v5` label identifies this training lineage; it is not a parameter-count
label.

## Architecture

| Field | Value |
|---|---:|
| Parameters | 756,819,456 |
| Layers | 24 |
| Hidden size | 1,536 |
| Attention heads | 12 (MHA) |
| FFN size | 4,096 (SwiGLU/SiLU) |
| Position encoding | RoPE, theta 10,000 |
| Normalization | RMSNorm |
| Embeddings | Tied |
| Tokenizer | GPT-2 BPE, vocabulary padded to 50,304 |
| Maximum context | 2,048 tokens |
| Published weight dtype | BF16 |

The Hugging Face architecture is `LlamaForCausalLM`, but the model uses the
GPT-2 tokenizer and the d24 architecture above. It is not a Llama-family
pretrained checkpoint.

## SFT data and training

The source conversations are the
[`sfanm/d24-sft-mixture`](https://huggingface.co/datasets/sfanm/d24-sft-mixture)
research mixture: SmolTalk, MMLU auxiliary data, GSM8K, spelling tasks, and
identity examples. The local simple-ChatML preparation produced 227,443 packed
training sequences and 7,381 packed validation sequences at length 2,048, with
about 476M tokens across both splits.

Training used:

| Hyperparameter | Value |
|---|---:|
| Epochs / steps | 1 / 1,777 |
| Global / micro batch | 128 / 1 |
| Sequence length | 2,048 |
| Optimizer | AdamW |
| Peak / minimum LR | 1e-4 / 1e-5 |
| LR schedule | Cosine, 50-step warmup |
| Adam betas / epsilon | (0.9, 0.95) / 1e-8 |
| Weight decay | 0.1 |
| Gradient clip | 1.0 |
| Precision | BF16 |

The final SFT validation language-model loss was **0.79675** (perplexity
**2.2183**). This is an in-distribution SFT validation metric, not a downstream
capability or safety evaluation.

## Chat format and stopping

The template is:

```text
<|im_start|>user
...<|im_end|>
<|im_start|>assistant
...
```

`<|im_end|>` is a literal string represented by seven GPT-2 BPE tokens. It is
**not** `eos_token_id=50256` and is not a registered special token. Generation
must stop on the string `<|im_end|>`; otherwise the model can continue into
another turn until `max_new_tokens`.

## Transformers usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sfanm/d24-sft-v5-simplechat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": "Natalia sold clips to 48 friends in April and half as many in May. How many total?",
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    stop_strings=["<|im_end|>"],
    tokenizer=tokenizer,
)
response = tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=False,
)
print(response)
```

For vLLM, use `SamplingParams(stop=["<|im_end|>"])`.

## Intended use

- research on SFT, RLVR, GRPO/RLOO/maxRL, and small-model behavior;
- a warm-start chat policy for the d24 regular-RL recipes;
- controlled comparisons with other d24 SFT checkpoints.

The accompanying `unified-training` repository contains the reproducible
cluster setup, current RL defaults, and algorithm baselines in
`docs/REGULAR_RL_TRAINING.md`.

## Limitations and evaluation caveats

- This is a small experimental research model and has not undergone production
  safety alignment or a comprehensive capability/safety evaluation.
- It can produce incorrect, incoherent, biased, or unsafe text. Verify outputs
  independently and do not use it for high-stakes decisions.
- The context window is 2,048 tokens.
- The SFT mixture includes GSM8K training examples. Do not treat GSM8K results
  from this checkpoint as a clean no-exposure benchmark. The separate local
  `d24-sft-v5-nogsm8k-hf` checkpoint exists for that control but is not part of
  this repository.
- Chat generation is incorrect unless the caller stops on the literal
  `<|im_end|>` string.
- The model and SFT mixture derive from multiple upstream datasets. The Hub
  metadata therefore uses `license: other`; review the source dataset licenses
  and terms before redistribution or downstream use.

## Reproducibility

- SFT checkpoint: `sft-d24-v5-simplechat/iter_0001777`
- HF export: `d24-sft-v5-simplechat-hf`
- Chat template: `src/nemotron/data_prep/templates/simple_chatml.jinja`
- Training repository: `yudasong/unified-training`