初始化项目,由ModelHub XC社区提供模型

Model: Adc05102002/bge-m3-vi-base
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-28 02:20:16 +08:00
commit 6641383a0a
6 changed files with 26762 additions and 0 deletions

61
README.md Normal file
View File

@@ -0,0 +1,61 @@
---
language:
- multilingual
license: mit
library_name: transformers
pipeline_tag: sentence-similarity
tags:
- embeddings
- text-embeddings-inference
- bge
- bge-m3
- multilingual
- retrieval
- semantic-search
- xlm-roberta
- custom-tokenizer
- long-context
base_model: BAAI/bge-m3
model_type: xlm-roberta
---
# BGE-M3 Custom Tokenizer (8.5K Vocab)
A customized version of :contentReference[oaicite:0]{index=0} with a newly trained tokenizer optimized for domain-specific multilingual retrieval workloads.
This model replaces the original XLM-R tokenizer vocabulary with a compact 8.5K-token tokenizer trained on a custom corpus.
## Highlights
- Based on `BAAI/bge-m3`
- Custom tokenizer trained from scratch
- Reduced vocabulary size: **8500**
- Long-context support: **8192 tokens**
- Multilingual retrieval and embedding model
- Optimized for:
- semantic search
- RAG pipelines
- dense retrieval
- domain-specific embeddings
---
# Model Details
## Base Model
- Architecture: XLM-RoBERTa
- Original model: `BAAI/bge-m3`
- Embedding dimension: 1024
- Transformer encoder model
## Tokenizer
The original tokenizer was replaced with a newly trained tokenizer using:
```python
tokenizer = base_tokenizer.train_new_from_iterator(
batch_iterator(),
vocab_size=8500,
min_frequency=2,
)