library_name, base_model, tags, license, datasets, language, pipeline_tag

library_name	base_model	tags	license	datasets	language	pipeline_tag
transformers	Qwen/Qwen3-8B-Base	multilingual reasoning LLM qwen3	apache-2.0	lightonai/Dolci-Think-SFT-32B-Multilingual	sw en	text-generation

Qwen3-8B-SW-Pivot-EN

Qwen3-8B-SW-Pivot-EN is an English-pivoted reasoning model fine-tuned from Qwen/Qwen3-8B-Base on Swahili questions and answers. This model receives questions in Swahili, produces its entire reasoning trace in English, then delivers the final answer in Swahili.

It is released alongside the paper Rethinking the Multilingual Reasoning Gap with Layer Swap.

Model details

• Base model: Qwen/Qwen3-8B-Base
• Language: Swahili (Q&A) with English CoT
• Training: Full SFT, ~10B tokens, 2 epochs
• Context length: 32,768 tokens
• Dataset: lightonai/Dolci-Think-SFT-32B-Multilingual (Swahili Q&A with English CoT).

Note

The model was trained on data derived from allenai/Dolci-Think-SFT-32B, released under the ODC-BY-1.0 license.

Related models

This model is part of a Swahili specialist trio designed to study the native reasoning gap:

Model	CoT language	Description
lightonai/Qwen3-8B-SW	Swahili	Native reasoning specialist
lightonai/Qwen3-8B-SW-Swap	Swahili	Layer Swap: middle layers (L13–L22) of Qwen3-8B-EN transplanted into Qwen3-8B-SW
lightonai/Qwen3-8B-SW-Pivot-EN	English	Same Swahili Q&A pairs, but CoT in English
lightonai/Qwen3-8B-EN	English	English specialist

Evaluation

All scores are mean accuracy (%) on the Swahili version of each benchmark, with sample standard deviation across runs. AIME 24/25 is averaged over 30 runs; the others over 10 runs, using the recommended generation parameters.

Model	MGSM-Rev2	Global-MMLU-Lite	GPQA-Diamond	AIME 24/25	HumanEvalPlus	Average
Qwen3-8B-SW	93.16	61.98	49.39	47.67	82.69	66.98
Qwen3-8B-SW-Swap	96.12	64.10	49.29	50.33	85.62	69.09
Qwen3-8B-SW-Pivot-EN	89.68	66.00	52.73	59.67	84.50	70.52
Qwen3-8B-EN	35.88	33.88	36.82	24.78	58.44	37.96

Benchmarks used:

• lightonai/gpqa_diamond_multilingual
• lightonai/aime24_multilingual
• lightonai/aime25_multilingual
• lightonai/HumanEvalPlus_multilingual
• lightonai/mgsm-rev2
• CohereLabs/Global-MMLU-Lite

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "lightonai/Qwen3-8B-SW-Pivot-EN"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Suluhisha: 24 × 17 = ?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)

outputs = model.generate(inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, top_k=20)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Recommended sampling: temperature=1.0, top_p=0.95, top_k=20, min_p=0.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{lasbordes2026rethinking,
  title        = {Rethinking the Multilingual Reasoning Gap with Layer Swap},
  author       = {Lasbordes, Maxence and Chatelain, Amélie and Seddah, Djamé},
  year         = {2026},
  eprint       = {2605.26735},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL}
}