77 lines
2.4 KiB
Markdown
77 lines
2.4 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model:
|
||
|
|
- tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2
|
||
|
|
library_name: mlx
|
||
|
|
tags:
|
||
|
|
- mlx
|
||
|
|
- quantized
|
||
|
|
- apple-silicon
|
||
|
|
- japanese
|
||
|
|
- swallow
|
||
|
|
language:
|
||
|
|
- ja
|
||
|
|
- en
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen3-Swallow-32B-RL-v0.2-MLX-fp16
|
||
|
|
|
||
|
|
This model is an [MLX](https://github.com/ml-explore/mlx) format conversion of [`tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2`](https://huggingface.co/tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2), optimized for Apple Silicon.
|
||
|
|
|
||
|
|
## Model Details
|
||
|
|
|
||
|
|
| Attribute | Value |
|
||
|
|
|---|---|
|
||
|
|
| **Original Model** | [`tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2`](https://huggingface.co/tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2) |
|
||
|
|
| **Architecture** | Dense Transformer |
|
||
|
|
| **Parameters** | 32B |
|
||
|
|
| **Quantization** | Full precision (fp16) |
|
||
|
|
| **Model Size** | ~61 GB |
|
||
|
|
| **Format** | MLX (Apple Silicon optimized) |
|
||
|
|
| **Converted with** | mlx-lm v0.30.8 |
|
||
|
|
| **License** | Apache 2.0 |
|
||
|
|
|
||
|
|
## About Qwen3-Swallow
|
||
|
|
|
||
|
|
Qwen3-Swallow is a bilingual Japanese-English large language model developed by the [Swallow Project](https://swallow-llm.github.io/) at the Institute of Science Tokyo (formerly Tokyo Institute of Technology) and AIST. Built upon Qwen3 through Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Reinforcement Learning (RL), it achieves strong performance on both Japanese and English tasks while maintaining capabilities in mathematics and coding.
|
||
|
|
|
||
|
|
For more details, see the [original model card](https://huggingface.co/tokyotech-llm/Qwen3-Swallow-32B-RL-v0.2).
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
### Quick Start (Python)
|
||
|
|
|
||
|
|
```python
|
||
|
|
from mlx_lm import load, generate
|
||
|
|
|
||
|
|
model, tokenizer = load("tocchitocchi/Qwen3-Swallow-32B-RL-v0.2-MLX-fp16")
|
||
|
|
|
||
|
|
messages = [{"role": "user", "content": "hello"}]
|
||
|
|
prompt = tokenizer.apply_chat_template(
|
||
|
|
messages, add_generation_prompt=True
|
||
|
|
)
|
||
|
|
|
||
|
|
response = generate(model, tokenizer, prompt=prompt, verbose=True, max_tokens=512)
|
||
|
|
```
|
||
|
|
|
||
|
|
### Interactive Chat
|
||
|
|
|
||
|
|
```bash
|
||
|
|
mlx_lm.chat --model tocchitocchi/Qwen3-Swallow-32B-RL-v0.2-MLX-fp16
|
||
|
|
```
|
||
|
|
|
||
|
|
### OpenAI-Compatible Server
|
||
|
|
|
||
|
|
```bash
|
||
|
|
mlx_lm.server --model tocchitocchi/Qwen3-Swallow-32B-RL-v0.2-MLX-fp16 --port 8080
|
||
|
|
```
|
||
|
|
|
||
|
|
Then connect with any OpenAI-compatible client at `http://localhost:8080/v1`.
|
||
|
|
|
||
|
|
## Acknowledgments
|
||
|
|
|
||
|
|
- Original model by [Swallow Project](https://swallow-llm.github.io/) (Institute of Science Tokyo and AIST)
|
||
|
|
- MLX framework by [Apple Machine Learning Research](https://github.com/ml-explore/mlx)
|
||
|
|
- Conversion performed using [mlx-lm](https://github.com/ml-explore/mlx-lm)
|