Rio-3.0-Open-Mini/README.md

---
license: mit
language:
- pt
- en
base_model:
- Qwen/Qwen3-4B-Thinking-2507
pipeline_tag: text-generation
library_name: transformers
---

# Rio 3.0 Open Mini

**Rio 3.0 Open Mini** is a frontier-class reasoning model developed by [IplanRIO](https://iplanrio.rio.rj.gov.br/), the municipal IT company of Rio de Janeiro's city government. Built through distillation on top of Qwen3-4B-Thinking-2507 using reasoning traces from our to be announced Rio 3.0 model, Rio 3.0 Open achieves state-of-the-art results across mathematics, STEM, and code benchmarks — surpassing its base model by significant margins and competing with models far larger than itself.

Rio 3.0 Open Mini features **SwiReasoning**, a training-free inference framework based on [Shi et al. (2025)](https://arxiv.org/abs/2510.05069) that dynamically switches between explicit chain-of-thought and latent-space reasoning, guided by entropy-based confidence signals. This enables both higher accuracy and dramatically improved token efficiency. This model was explicitly trained to maximize the efficiency gained via latent reasoning.

## Key Features

- **4B total parameters**
- **262,144 token context window**
- **SwiReasoning integration** — dynamic explicit/latent reasoning switching for Pareto-superior accuracy and efficiency
- **Distilled from Qwen3-4B-Thinking-2507 with traces from Rio 3.0**
- **Multilingual** — strong performance in Portuguese, English, Chinese, and dozens of other languages
- **MIT License** — fully open for commercial and research use

## Benchmark Results

### Mathematics & STEM

| Model | GPQA Diamond | LiveCodeBench | Composite Math* | AIME 2025 | AIME 2026 I | HMMT 2025 I | HMMT 2025 II | BRUMO 2025 | CMIMC 2025 | SMT 2025 |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **Rio 3.0 Open Mini** | **71.90%** | **63.50%** | **78.11%** | **89.17%** | **75.00%** | **73.33%** | **79.17%** | **85.83%** | **66.88%** | **77.36%** |
| Rio 3.0 Open Mini (w/o latent) | 70.10% | 62.00% | 75.53% | 85.83% | 75.83% | 66.67% | 74.17% | 84.17% | 63.75% | 78.30% |
| Qwen3-4B-2507 (base) | 65.80% | 55.20% | 71.12% | 81.67% | 70.83% | 55.83% | 73.33% | 81.67% | 60.00% | 74.53% |
| Qwen3-30B-A3B-2507 | 73.40% | 66.00% | 76.08% | 82.50% | 76.67% | 70.83% | 75.83% | 85.00% | 66.25% | 75.47% |
| GPT OSS 20B | 71.50% | 70.26% | 82.34% | 89.17% | 85.00% | 76.67% | 83.33% | 86.67% | 72.50% | 83.02% |

*Composite Math is the average across all other mathematics benchmarks in this table.

### Rio Model Family Comparison

| Model | GPQA Diamond | LiveCodeBench | Composite Math* | AIME 2025 |
|:---|:---:|:---:|:---:|:---:|
| **Rio 3.0 Open** | **85.10%** | **76.00%** | **91.78%** | **96.67%** |
| Rio 2.5 Open | 77.20% | 69.60% | 87.53% | 93.33% |
| Rio 3.0 Open Mini | 71.90% | 63.50% | 78.11% | 89.17% |

### Gains Over Base Model (Qwen3-4B-2507)
| Benchmark | Base Model | Rio 3.0 Open Mini | Δ |
|:---|:---:|:---:|:---:|
| GPQA Diamond | 65.80% | 71.90% | **+6.10%** |
| LiveCodeBench | 55.20% | 63.50% | **+8.30%** |
| Composite Math | 71.12% | 78.11% | **+6.99%** |
| AIME 2025 | 81.67% | 89.17% | **+7.50%** |
| AIME 2026 I | 70.83% | 75.00% | **+4.17%** |
| HMMT 2025 I | 55.83% | 73.33% | **+17.50%** |
| BRUMO 2025 | 81.67% | 85.83% | **+4.16%** |
| CMIMC 2025 | 60.00% | 66.88% | **+6.88%** |
| SMT 2025 | 74.53% | 77.36% | **+2.83%** |

## SwiReasoning: Latent/Explicit Reasoning

Rio 3.0 Open Mini integrates [SwiReasoning](https://arxiv.org/abs/2510.05069) (Shi et al., 2025), a training-free inference framework that dynamically alternates between two reasoning modes:

- **Explicit reasoning** — standard chain-of-thought in natural language, where the model commits tokens to a single reasoning path
- **Latent reasoning** — continuous reasoning in hidden space, where the model explores multiple implicit paths simultaneously without emitting tokens

The switching is governed by **block-wise confidence** estimated from entropy trends in the next-token distribution. When confidence is low (entropy trending upward), the model enters latent mode to explore alternatives. When confidence recovers, it switches back to explicit mode to commit to a solution.

This approach achieves a **Pareto-superior** trade-off: higher accuracy at unlimited budgets *and* dramatically better token efficiency under constrained budgets.

The benchmark table above includes **(w/o latent)** rows showing performance with standard explicit-only reasoning, demonstrating the consistent gains from SwiReasoning across all benchmarks.

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prefeitura-rio/Rio-3.0-Open-Mini"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Write a poem about Rio de Janeiro."

messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=81920,
    temperature=0.6,
    top_p=0.95,
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
print(response)
```

### Using with vLLM

```bash
vllm serve prefeitura-rio/Rio-3.0-Open-Mini \
    --tensor-parallel-size 4 \
    --max-model-len 262144 \
    --trust-remote-code
```

### Using with SGLang

```bash
python -m sglang.launch_server \
    --model-path prefeitura-rio/Rio-3.0-Open-Mini \
    --tp 4 \
    --context-length 262144 \
    --trust-remote-code
```

## Model Details

| | |
|:---|:---|
| **Developer** | IplanRIO — Empresa Municipal de Informática e Planejamento S.A. |
| **Base Model** | Qwen3-4B-Thinking-2507 |
| **Architecture** | Transformer |
| **Total Parameters** | ~4B |
| **Context Length** | 262,144 tokens |
| **Default Max Output Length** | 81,920 tokens |
| **Training Method** | Distillation |
| **Inference Enhancement** | SwiReasoning (latent/explicit switching) |
| **License** | MIT |
| **Languages** | Multilingual (en, pt, zh, ja, ko, fr, de, es, ar, and more) |

## Citation

If you use SwiReasoning, please also cite:

```bibtex
@misc{shi2025swireasoning,
    title={SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs},
    author={Dachuan Shi et al.},
    year={2025},
    eprint={2510.05069},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

## Acknowledgments

Rio 3.0 Open Mini is built upon the exceptional work of the [Qwen Team](https://github.com/QwenLM) and their Qwen3 model family. We also acknowledge the authors of [SwiReasoning](https://github.com/sdc17/SwiReasoning) for their innovative inference framework.

Developed in Rio de Janeiro 🇧🇷 by [IplanRIO](https://iplanrio.rio.rj.gov.br/).