---
language:
- multilingual
license: mit
library_name: transformers
pipeline_tag: sentence-similarity
tags:
- embeddings
- text-embeddings-inference
- bge
- bge-m3
- multilingual
- retrieval
- semantic-search
- xlm-roberta
- custom-tokenizer
- long-context
base_model: BAAI/bge-m3
model_type: xlm-roberta
---

# BGE-M3 Custom Tokenizer (8.5K Vocab)

A customized version of :contentReference[oaicite:0]{index=0} with a newly trained tokenizer optimized for domain-specific multilingual retrieval workloads.

This model replaces the original XLM-R tokenizer vocabulary with a compact 8.5K-token tokenizer trained on a custom corpus.

## Highlights

- Based on `BAAI/bge-m3`
- Custom tokenizer trained from scratch
- Reduced vocabulary size: **8500**
- Long-context support: **8192 tokens**
- Multilingual retrieval and embedding model
- Optimized for:
  - semantic search
  - RAG pipelines
  - dense retrieval
  - domain-specific embeddings

---

# Model Details

## Base Model

- Architecture: XLM-RoBERTa
- Original model: `BAAI/bge-m3`
- Embedding dimension: 1024
- Transformer encoder model

## Tokenizer

The original tokenizer was replaced with a newly trained tokenizer using:

```python
tokenizer = base_tokenizer.train_new_from_iterator(
    batch_iterator(),
    vocab_size=8500,
    min_frequency=2,
)