gpt-oss-15.5b-23E-SFT/README.md

license, library_name, pipeline_tag, tags, base_model

license	library_name	pipeline_tag	tags	base_model
apache-2.0	transformers	text-generation	gpt-oss mixture-of-experts moe reasoning sft science	openai/gpt-oss

gpt-oss-15.5b-23E-SFT

A supervised fine-tuned (SFT) variant of a gpt_oss Mixture-of-Experts model, specialized for science-domain reasoning. The model is a pruned/customized configuration with 23 local experts (top-4 routing) and approximately 15.5B total parameters.

Model details

Architecture	GptOssForCausalLM (Mixture-of-Experts)
Hidden size	2880
Layers	24 (alternating sliding / full attention)
Attention heads	64 (8 KV heads, GQA)
Local experts	23
Experts per token	4
Vocab size	201,088
Max position embeddings	131,072 (YaRN, factor 32× from 4096)
Sliding window	128
dtype	bfloat16

See config.json for the full configuration.

Training

This checkpoint corresponds to step 1175 of an SFT run with the following setup:

• Base: gpt_oss 23-expert variant
• Data: ~30k science / reasoning samples
• Epochs: 5
• Global batch size: 128
• Learning rate: 1e-5
• Precision: bfloat16

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "xpuenabler/gpt-oss-15.5b-23E-SFT"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain why the sky is blue."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

A custom chat_template.jinja is bundled with the tokenizer and is applied automatically by apply_chat_template.

Files

• model.safetensors — model weights (bfloat16, ~31 GB)
• config.json, generation_config.json — model / generation configuration
• tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja — tokenizer assets

Intended use & limitations

This is a research checkpoint intended for science-oriented reasoning experiments. It has not been aligned for safety beyond what was inherited from the base model and the SFT data, and may produce inaccurate or unsafe outputs. Evaluate carefully before any downstream use.

License

Released under the Apache 2.0 license, consistent with the upstream gpt-oss family.