BGE M3

According to NVIDIA's NIM reference page, BAAI/bge-m3 is a multilingual embedding model that simultaneously performs dense retrieval, multi-vector retrieval, and sparse retrieval across more than 100 working languages, with a maximum input of 8192 tokens spanning short sentences to long documents. The model outputs 1024-dimensional embeddings and is offered under the MIT License. The documentation recommends a hybrid retrieval plus re-ranking pipeline (combining embedding retrieval with BM25-style scoring or a cross-encoder reranker), and lists related BGE variants including bge-m3-unsupervised, bge-m3-retromae, and English-only bge-large/base/small-en-v1.5 models.

The official BGE documentation page for BGE-M3 describes it as a 569-million-parameter, 2.27 GB multilingual embedding model with multi-functionality, multi-linguality, and multi-granularity. Training covered 170+ languages, though the paper reports 100+ working languages with uneven per-language data. The page extends max position to 8192 tokens and introduces MCLS (Multiple CLS), a method that enhances long-text embedding without additional fine-tuning. These details come from bge-model.com, BAAI's official documentation site. Technical implementation details include dense retrieval via normalized [CLS] hidden states, sparse retrieval using a linear layer plus ReLU activation over encoder outputs to produce term weights, and multi-vector (ColBERT-style) scoring. The relevance score for sparse retrieval is computed as the sum of joint importance weights over co-occurring query and passage terms. The page positions BGE-M3 as the compound flagship of the BGE embedding family, unifying the three retrieval paradigms that earlier BGE models handled separately.

The BAAI/bge-m3 Hugging Face model card from BAAI's FlagOpen project describes BGE-M3's multi-functionality (dense, multi-vector, and sparse retrieval in one model), multi-linguality (100+ languages), and multi-granularity (up to 8192 tokens). It recommends a hybrid retrieval plus re-ranking pipeline, noting that BGE-M3 outputs both dense embeddings and BM25-like sparse token weights simultaneously, with Vespa and Milvus cited as integration targets. Re-ranker models such as bge-reranker and bge-reranker-v2 are suggested for post-retrieval filtering. The card links to the paper and the FlagEmbedding GitHub repository for code. News items on the model card include a July 1, 2024 update correcting BGE-M3's MIRACL evaluation results: prior numbers were lower due to a query-id filtering mistake, and corrected results are higher though conclusions are unchanged. A March 20, 2024 entry notes Milvus team support enabling hybrid retrieval of bge-m3 via a PyMilvus example. An earlier March 8, 2024 note acknowledges community experimental results. These updates confirm BGE-M3's continued maintenance and ecosystem integration as of mid-2024.

The original BGE M3-Embedding paper (arXiv:2402.03216v3, February 2024) by Chen, Xiao, Zhang, Luo, and Lian from BAAI and the University of Science and Technology of China introduces the M3-Embedding model. The paper establishes BGE-M3's three core capabilities: multi-linguality across more than 100 working languages, multi-functionality (simultaneous dense, multi-vector, and sparse retrieval), and multi-granularity spanning short sentences to 8192-token documents. Training contributions include a novel self-knowledge distillation approach that integrates relevance scores across retrieval functionalities, plus an optimized batching strategy for discriminative embeddings. The authors note the model and code are publicly available via the FlagOpen/FlagEmbedding GitHub repository. To the best of the authors' knowledge, M3-Embedding is the first embedding model to realize this combined versatility across dense, multi-vector (ColBERT-style), and sparse retrieval in a single unified model. This unified foundation supports real-world IR applications without requiring separate models per retrieval paradigm. The paper positions BGE-M3 as setting new state-of-the-art results on multi-lingual and cross-lingual retrieval benchmarks. The work provides the technical basis for subsequent integrations such as hybrid retrieval pipelines in Vespa and Milvus.