160 lines
7.5 KiB
Markdown
160 lines
7.5 KiB
Markdown
---
|
|
library_name: transformers
|
|
license: llama3.2
|
|
base_model: scb10x/llama3.2-typhoon2-3b-instruct
|
|
model-index:
|
|
- name: llama3.2-typhoon2-3b-instruct
|
|
results: []
|
|
language:
|
|
- en
|
|
- th
|
|
datasets:
|
|
- scb10x/typhoon-t1-3b-research-preview-data
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Typhoon T1 3B (Research Preview)
|
|
|
|
**Typhoon T1 3B (Research Preview)** is the first in a new family of open reasoning model "**Typhoon T**". Reasoning model is a novel type of model that think longer before giving a final answer.
|
|
|
|
Typhoon T1 3B (Research Preview) is built on top of [Typhoon 2 3B Instruct](https://huggingface.co/scb10x/llama3.2-typhoon2-3b-instruct). It has improved performance on challenging benchmarks like GPQA, MMLU Pro, and AI Mathematics Olympiad validation set.
|
|
|
|
> [!NOTE]
|
|
> 🚀
|
|
> **2025-02-01 Update**: We have released a new version of Typhoon T1 3B (Research Preview) with the ability to 🇹🇭 *generate Thai reasoning traces*, improved *Thai performance in general*, and *enhanced instruction following*. This version has the a comparative level of English performance to `v2025-01-23`.
|
|
|
|
## Key Points
|
|
|
|
- **Typhoon T1** is a new family of open reasoning models developed by SCB 10X
|
|
- **Typhoon T1 3B (Research Preview)**, the *first* in the **Typhoon T** family, shows improved performance across **challenging benchmarks** compared to the original Typhoon 2 3B Instruct
|
|
- Typhoon T1 3B (Research Preview) offers a **fast**, **low-compute requirements** model, yet is **capable** in a variety of tasks by scaling test-time compute, enabling the model to think longer before giving a final answer. Typhoon T1 3B (Research Preview) is able to **_reason across domains_**, unlike many open reasoning models limited to mathematics and coding
|
|
- We **open** our recipe for data pipeline and training this model without distilling from other reasoning models
|
|
- We introduce **a new thinking paradigm** for reasoning models, structured thinking, where we add auxiliary tokens to help structure the thinking process of the model. This approach shows an increase in performance over a common variant of separating only thought and response parts based on our experiments
|
|
- Typhoon T1 3B (Research Preview) `v2025-02-01` is the first reasoning model where we intentionally equipped the model with the ability to **generate Thai reasoning traces**, improving *transparency* and *interpretability* of the model.
|
|
|
|
For more technical details, please visit our [technical blog](https://blog.opentyphoon.ai/introducing-typhoon-t1-a-family-of-open-reasoning-models-research-preview-22daacc88662).
|
|
|
|
* To acknowledge Meta's effort in creating the foundation model and to comply with the license, we explicitly include `llama-3.2` in the model name.
|
|
|
|
## Performance
|
|
|
|
| Model name | GSM8K (↑), 8-shot | HumanEval+ (↑), Pass@10 | GPQA (↑), 0CoT | AIME (↑) |
|
|
|-----------------------|-------------------|-------------------------|----------------|----------|
|
|
| Typhoon 2 3B Instruct | 56.63 | 66 | 27.01 | 0 |
|
|
| Typhoon T1 3B (semi) | 59.59 | 68.99 | 25.89 | 0 |
|
|
| **Typhoon T1 3B (Research Preview)** v2025-01-23 | **62.40** | **69.87** | **31.7** | **2.22** |
|
|
|
|
### MMLU Pro (↑), 5-shot
|
|
|
|
| Model name | Average | Math | Health | Physics | Business | Biology | Chemistry | Computer Science | Economics | Engineering | Philosophy | Other | History | Psychology | Law |
|
|
|-----------------------|-----------|-----------|-----------|----------|-----------|-----------|-----------|------------------|-----------|-------------|------------|-----------|-----------|------------|-----------|
|
|
| Typhoon 2 3B Instruct | 26.7 | 26.8 | 33.62 | 23.4 | 25.35 | 43.38 | 19.88 | 28.29 | 35.43 | 18.37 | 28.06 | 27.92 | 25.72 | 37.84 | 13.17 |
|
|
| **Typhoon T1 3B (Research Preview)** v2025-01-23 | **30.65** | **30.57** | **36.19** | **27.1** | **31.69** | **50.77** | **22.17** | **31.22** | **38.86** | **21.98** | **30.66** | **32.79** | **26.51** | **43.36** | **17.26** |
|
|
|
|
## Model description
|
|
|
|
- **Model type**: A 3B instruct decoder-only model based on Llama architecture.
|
|
- **Requirement**: `transformers` 4.46.1 or newer.
|
|
- **Primary Language(s)**: English 🇬🇧 and Thai 🇹🇭
|
|
- **License**: [Llama 3.2 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE)
|
|
|
|
|
|
## Usage Examples
|
|
|
|
⚠️ Please note that `max_new_tokens` should be at least `512`, but is recommended at a minimum of `1,024` to provide space for complete generation.
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
import torch
|
|
|
|
model_id = "scb10x/llama-3.2-typhoon-t1-3b-research-preview"
|
|
revision = "main" # To use the previous version comment this line
|
|
# revision = "v2025-01-23" # To use the previous version uncomment this line
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
torch_dtype=torch.bfloat16,
|
|
device_map="auto",
|
|
revision=revision
|
|
)
|
|
|
|
messages = [
|
|
{"role": "user", "content": "หากแปลคำว่า \"ไต้ฝุ่น\" เป็นภาษาอังกฤษ ในคำที่ถูกแปลแล้วจะมีตัวอักษร \"o\" ทั้งหมดกี่ตัว"},
|
|
]
|
|
|
|
input_ids = tokenizer.apply_chat_template(
|
|
messages,
|
|
add_generation_prompt=True,
|
|
return_tensors="pt"
|
|
).to(model.device)
|
|
|
|
terminators = [
|
|
tokenizer.eos_token_id,
|
|
tokenizer.convert_tokens_to_ids("<|eot_id|>")
|
|
]
|
|
|
|
outputs = model.generate(
|
|
input_ids,
|
|
max_new_tokens=1024,
|
|
eos_token_id=terminators,
|
|
do_sample=False,
|
|
temperature=0.0,
|
|
top_p=0.9,
|
|
)
|
|
response = outputs[0][input_ids.shape[-1]:]
|
|
print(tokenizer.decode(response, skip_special_tokens=True))
|
|
```
|
|
|
|
### OpenAI API-compatible Server with vLLM
|
|
|
|
```bash
|
|
pip install vllm
|
|
vllm serve scb10x/llama-3.2-typhoon-t1-3b-research-preview
|
|
|
|
# To serve the previous version, add the revision parameter as shown below
|
|
# vllm serve scb10x/llama-3.2-typhoon-t1-3b-research-preview --revision v2025-01-23
|
|
# see more information at https://docs.vllm.ai/
|
|
```
|
|
|
|
## Intended uses & limitations
|
|
|
|
While we made an effort to make our model safe, like all generative models, it may generate unsafe content in rare cases. Introducing a reasoning model paradigm may introduce some unforeseen behaviors, as model safety in the reasoning domain is a relatively new and ongoing area of research.
|
|
|
|
## Follow Us
|
|
|
|
[https://twitter.com/opentyphoon](https://twitter.com/opentyphoon)
|
|
|
|
## Support
|
|
|
|
[https://discord.gg/us5gAYmrxw](https://discord.gg/us5gAYmrxw)
|
|
|
|
## Training hyperparameters
|
|
|
|
The following hyperparameters were used during training:
|
|
- learning_rate: 2e-05
|
|
- train_batch_size: 6
|
|
- eval_batch_size: 1
|
|
- seed: 42
|
|
- distributed_type: multi-GPU
|
|
- num_devices: 8
|
|
- gradient_accumulation_steps: 6
|
|
- total_train_batch_size: 288
|
|
- total_eval_batch_size: 8
|
|
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08
|
|
- lr_scheduler_type: cosine
|
|
- num_epochs: 2.0
|
|
|
|
## Citation
|
|
|
|
```
|
|
@misc{taveekitworachai2025typhoont1openthai,
|
|
title={Typhoon T1: An Open Thai Reasoning Model},
|
|
author={Pittawat Taveekitworachai and Potsawee Manakul and Kasima Tharnpipitchai and Kunat Pipatanakul},
|
|
year={2025},
|
|
eprint={2502.09042},
|
|
archivePrefix={arXiv},
|
|
primaryClass={cs.CL},
|
|
url={https://arxiv.org/abs/2502.09042},
|
|
}
|
|
``` |