初始化项目,由ModelHub XC社区提供模型

Model: xpuenabler/gpt-oss-15.5b-23E-SFT
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-10-06 12:27:18 +08:00
commit c0a44473f3
9 changed files with 751 additions and 0 deletions

89
README.md Normal file
View File

@@ -0,0 +1,89 @@
---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- gpt-oss
- mixture-of-experts
- moe
- reasoning
- sft
- science
base_model: openai/gpt-oss
---
# gpt-oss-15.5b-23E-SFT
A supervised fine-tuned (SFT) variant of a `gpt_oss` Mixture-of-Experts model, specialized for science-domain reasoning. The model is a pruned/customized configuration with **23 local experts** (top-4 routing) and approximately **15.5B total parameters**.
## Model details
| | |
|---|---|
| Architecture | `GptOssForCausalLM` (Mixture-of-Experts) |
| Hidden size | 2880 |
| Layers | 24 (alternating sliding / full attention) |
| Attention heads | 64 (8 KV heads, GQA) |
| Local experts | 23 |
| Experts per token | 4 |
| Vocab size | 201,088 |
| Max position embeddings | 131,072 (YaRN, factor 32× from 4096) |
| Sliding window | 128 |
| dtype | bfloat16 |
See `config.json` for the full configuration.
## Training
This checkpoint corresponds to **step 1175** of an SFT run with the following setup:
- **Base**: `gpt_oss` 23-expert variant
- **Data**: ~30k science / reasoning samples
- **Epochs**: 5
- **Global batch size**: 128
- **Learning rate**: 1e-5
- **Precision**: bfloat16
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "xpuenabler/gpt-oss-15.5b-23E-SFT"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo,
dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "user", "content": "Explain why the sky is blue."},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```
A custom `chat_template.jinja` is bundled with the tokenizer and is applied automatically by `apply_chat_template`.
## Files
- `model.safetensors` — model weights (bfloat16, ~31 GB)
- `config.json`, `generation_config.json` — model / generation configuration
- `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.jinja` — tokenizer assets
## Intended use & limitations
This is a research checkpoint intended for science-oriented reasoning experiments. It has not been aligned for safety beyond what was inherited from the base model and the SFT data, and may produce inaccurate or unsafe outputs. Evaluate carefully before any downstream use.
## License
Released under the Apache 2.0 license, consistent with the upstream `gpt-oss` family.