---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - gpt-oss
  - mixture-of-experts
  - moe
  - reasoning
  - sft
  - science
base_model: openai/gpt-oss
---

# gpt-oss-15.5b-23E-SFT

A supervised fine-tuned (SFT) variant of a `gpt_oss` Mixture-of-Experts model, specialized for science-domain reasoning. The model is a pruned/customized configuration with **23 local experts** (top-4 routing) and approximately **15.5B total parameters**.

## Model details

| | |
|---|---|
| Architecture | `GptOssForCausalLM` (Mixture-of-Experts) |
| Hidden size | 2880 |
| Layers | 24 (alternating sliding / full attention) |
| Attention heads | 64 (8 KV heads, GQA) |
| Local experts | 23 |
| Experts per token | 4 |
| Vocab size | 201,088 |
| Max position embeddings | 131,072 (YaRN, factor 32× from 4096) |
| Sliding window | 128 |
| dtype | bfloat16 |

See `config.json` for the full configuration.

## Training

This checkpoint corresponds to **step 1175** of an SFT run with the following setup:

- **Base**: `gpt_oss` 23-expert variant
- **Data**: ~30k science / reasoning samples
- **Epochs**: 5
- **Global batch size**: 128
- **Learning rate**: 1e-5
- **Precision**: bfloat16

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "xpuenabler/gpt-oss-15.5b-23E-SFT"

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain why the sky is blue."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```

A custom `chat_template.jinja` is bundled with the tokenizer and is applied automatically by `apply_chat_template`.

## Files

- `model.safetensors` — model weights (bfloat16, ~31 GB)
- `config.json`, `generation_config.json` — model / generation configuration
- `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.jinja` — tokenizer assets

## Intended use & limitations

This is a research checkpoint intended for science-oriented reasoning experiments. It has not been aligned for safety beyond what was inherited from the base model and the SFT data, and may produce inaccurate or unsafe outputs. Evaluate carefully before any downstream use.

## License

Released under the Apache 2.0 license, consistent with the upstream `gpt-oss` family.