prefeitura-rio/Rio-3.0-Open-Mini
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: prefeitura-rio/Rio-3.0-Open-Mini
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-18 23:21:02 +08:00
commit 266e5464f8
9 changed files with 151874 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

162
README.md Normal file
View File

@@ -0,0 +1,162 @@
---
license: mit
language:
- pt
- en
base_model:
- Qwen/Qwen3-4B-Thinking-2507
pipeline_tag: text-generation
library_name: transformers
---
# Rio 3.0 Open Mini
**Rio 3.0 Open Mini** is a frontier-class reasoning model developed by [IplanRIO](https://iplanrio.rio.rj.gov.br/), the municipal IT company of Rio de Janeiro's city government. Built through distillation on top of Qwen3-4B-Thinking-2507 using reasoning traces from our to be announced Rio 3.0 model, Rio 3.0 Open achieves state-of-the-art results across mathematics, STEM, and code benchmarks — surpassing its base model by significant margins and competing with models far larger than itself.
Rio 3.0 Open Mini features **SwiReasoning**, a training-free inference framework based on [Shi et al. (2025)](https://arxiv.org/abs/2510.05069) that dynamically switches between explicit chain-of-thought and latent-space reasoning, guided by entropy-based confidence signals. This enables both higher accuracy and dramatically improved token efficiency. This model was explicitly trained to maximize the efficiency gained via latent reasoning.
## Key Features
- **4B total parameters**
- **262,144 token context window**
- **SwiReasoning integration** — dynamic explicit/latent reasoning switching for Pareto-superior accuracy and efficiency
- **Distilled from Qwen3-4B-Thinking-2507 with traces from Rio 3.0**
- **Multilingual** — strong performance in Portuguese, English, Chinese, and dozens of other languages
- **MIT License** — fully open for commercial and research use
## Benchmark Results
### Mathematics & STEM
| Model | GPQA Diamond | LiveCodeBench | Composite Math* | AIME 2025 | AIME 2026 I | HMMT 2025 I | HMMT 2025 II | BRUMO 2025 | CMIMC 2025 | SMT 2025 |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **Rio 3.0 Open Mini** | **71.90%** | **63.50%** | **78.11%** | **89.17%** | **75.00%** | **73.33%** | **79.17%** | **85.83%** | **66.88%** | **77.36%** |
| Rio 3.0 Open Mini (w/o latent) | 70.10% | 62.00% | 75.53% | 85.83% | 75.83% | 66.67% | 74.17% | 84.17% | 63.75% | 78.30% |
| Qwen3-4B-2507 (base) | 65.80% | 55.20% | 71.12% | 81.67% | 70.83% | 55.83% | 73.33% | 81.67% | 60.00% | 74.53% |
| Qwen3-30B-A3B-2507 | 73.40% | 66.00% | 76.08% | 82.50% | 76.67% | 70.83% | 75.83% | 85.00% | 66.25% | 75.47% |
| GPT OSS 20B | 71.50% | 70.26% | 82.34% | 89.17% | 85.00% | 76.67% | 83.33% | 86.67% | 72.50% | 83.02% |
*Composite Math is the average across all other mathematics benchmarks in this table.
### Rio Model Family Comparison
| Model | GPQA Diamond | LiveCodeBench | Composite Math* | AIME 2025 |
|:---|:---:|:---:|:---:|:---:|
| **Rio 3.0 Open** | **85.10%** | **76.00%** | **91.78%** | **96.67%** |
| Rio 2.5 Open | 77.20% | 69.60% | 87.53% | 93.33% |
| Rio 3.0 Open Mini | 71.90% | 63.50% | 78.11% | 89.17% |
### Gains Over Base Model (Qwen3-4B-2507)
| Benchmark | Base Model | Rio 3.0 Open Mini | Δ |
|:---|:---:|:---:|:---:|
| GPQA Diamond | 65.80% | 71.90% | **+6.10%** |
| LiveCodeBench | 55.20% | 63.50% | **+8.30%** |
| Composite Math | 71.12% | 78.11% | **+6.99%** |
| AIME 2025 | 81.67% | 89.17% | **+7.50%** |
| AIME 2026 I | 70.83% | 75.00% | **+4.17%** |
| HMMT 2025 I | 55.83% | 73.33% | **+17.50%** |
| BRUMO 2025 | 81.67% | 85.83% | **+4.16%** |
| CMIMC 2025 | 60.00% | 66.88% | **+6.88%** |
| SMT 2025 | 74.53% | 77.36% | **+2.83%** |
## SwiReasoning: Latent/Explicit Reasoning
Rio 3.0 Open Mini integrates [SwiReasoning](https://arxiv.org/abs/2510.05069) (Shi et al., 2025), a training-free inference framework that dynamically alternates between two reasoning modes:
- **Explicit reasoning** — standard chain-of-thought in natural language, where the model commits tokens to a single reasoning path
- **Latent reasoning** — continuous reasoning in hidden space, where the model explores multiple implicit paths simultaneously without emitting tokens
The switching is governed by **block-wise confidence** estimated from entropy trends in the next-token distribution. When confidence is low (entropy trending upward), the model enters latent mode to explore alternatives. When confidence recovers, it switches back to explicit mode to commit to a solution.
This approach achieves a **Pareto-superior** trade-off: higher accuracy at unlimited budgets *and* dramatically better token efficiency under constrained budgets.
The benchmark table above includes **(w/o latent)** rows showing performance with standard explicit-only reasoning, demonstrating the consistent gains from SwiReasoning across all benchmarks.
## How to Use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "prefeitura-rio/Rio-3.0-Open-Mini"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
prompt = "Write a poem about Rio de Janeiro."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=81920,
temperature=0.6,
top_p=0.95,
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
print(response)
```
### Using with vLLM
```bash
vllm serve prefeitura-rio/Rio-3.0-Open-Mini \
--tensor-parallel-size 4 \
--max-model-len 262144 \
--trust-remote-code
```
### Using with SGLang
```bash
python -m sglang.launch_server \
--model-path prefeitura-rio/Rio-3.0-Open-Mini \
--tp 4 \
--context-length 262144 \
--trust-remote-code
```
## Model Details
| | |
|:---|:---|
| **Developer** | IplanRIO — Empresa Municipal de Informática e Planejamento S.A. |
| **Base Model** | Qwen3-4B-Thinking-2507 |
| **Architecture** | Transformer |
| **Total Parameters** | ~4B |
| **Context Length** | 262,144 tokens |
| **Default Max Output Length** | 81,920 tokens |
| **Training Method** | Distillation |
| **Inference Enhancement** | SwiReasoning (latent/explicit switching) |
| **License** | MIT |
| **Languages** | Multilingual (en, pt, zh, ja, ko, fr, de, es, ar, and more) |
## Citation
If you use SwiReasoning, please also cite:
```bibtex
@misc{shi2025swireasoning,
title={SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs},
author={Dachuan Shi et al.},
year={2025},
eprint={2510.05069},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
## Acknowledgments
Rio 3.0 Open Mini is built upon the exceptional work of the [Qwen Team](https://github.com/QwenLM) and their Qwen3 model family. We also acknowledge the authors of [SwiReasoning](https://github.com/sdc17/SwiReasoning) for their innovative inference framework.
Developed in Rio de Janeiro 🇧🇷 by [IplanRIO](https://iplanrio.rio.rj.gov.br/).

30
config.json Normal file
View File

@@ -0,0 +1,30 @@
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 2560,
"initializer_range": 0.02,
"intermediate_size": 9728,
"max_position_embeddings": 262144,
"max_window_layers": 36,
"model_type": "qwen3",
"num_attention_heads": 32,
"num_hidden_layers": 36,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 5000000,
"sliding_window": null,
"tie_word_embeddings": true,
"torch_dtype": "bfloat16",
"transformers_version": "4.51.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

13
generation_config.json Normal file
View File

@@ -0,0 +1,13 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95,
"transformers_version": "4.51.0"
}

151387
merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5bbf6d932270dc07b884d0cdf81dd80132303a849de7a70761808217b01a8867
size 8044982080

BIN
tokenizer.json (Stored with Git LFS) Normal file
View File

Binary file not shown.

239
tokenizer_config.json Normal file
View File

@@ -0,0 +1,239 @@
{
"add_prefix_space": false,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151665": {
"content": "<tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151666": {
"content": "</tool_response>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151667": {
"content": "<think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151668": {
"content": "</think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"bos_token": null,
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n<think>\\n' }}\n{%- endif %}",
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"model_max_length": 262144,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null,
"add_bos_token": false
}

1
vocab.json Normal file
View File

File diff suppressed because one or more lines are too long