BKAI's Vietnamese Bi-Encoder Hits 824K Downloads Beating Multilingual Models
A PhoBERT-based Vietnamese sentence embedding model from BKAI is quietly powering RAG and semantic search apps with over 800K downloads.
- BKAI's vietnamese-bi-encoder hits 824K+ downloads as a go-to Vietnamese embedding model
- Built on PhoBERT-base-v2, outputs 768-dim embeddings, ~0.1B parameters, Apache-2.0 licensed
- Trained on translated MS MARCO, SQuAD v2, and 80% of Zalo 2021 Legal Retrieval challenge
- Scores 73.28 Acc@1 and 80.73 MRR@10 on held-out Zalo legal retrieval test
- Requires word-segmented input via pyvi, underthesea, or RDRSegmenter before tokenization
- Powers 53+ Spaces and 62 fine-tuned derivatives, mostly Vietnamese RAG and legal chatbots
BKAI-HUST Foundation Models Lab’s Vietnamese bi-encoder has exceeded 824,000 downloads on Hugging Face. The model combines Vietnamese-specific retrieval training, a compact footprint, independent query and document encoding, and an Apache 2.0 license. That profile suits self-hosted semantic search and retrieval-augmented generation (RAG), where a retriever selects source text for a language model.
Many prominent embedding families prioritize English or distribute training across dozens of languages. BKAI’s checkpoint targets Vietnamese with a monolingual backbone and Vietnamese retrieval data, giving developers an alternative to hosted embedding APIs and generic multilingual models.
Independent encoders keep retrieval fast
The checkpoint turns PhoBERT-base-v2, VinAI’s Vietnamese RoBERTa model, into a Sentence Transformers bi-encoder with the following operating profile:
- Output: One 768-dimensional dense vector per sentence or passage
- Similarity: Cosine similarity between query and document vectors
- Maximum input: 256 model tokens
- Size: Roughly 135 million parameters
- License: Apache 2.0, including commercial use
A bi-encoder processes queries and documents independently. Document vectors can therefore be computed once and stored in a vector database, while only the incoming query needs encoding at runtime. A cross-encoder reranker can inspect the highest-ranked query-passage pairs when additional precision justifies the extra latency.
BKAI trained the model for 15 epochs on Vietnamese translations of MS MARCO and SQuAD v2, plus the 80% training split from the Legal Text Retrieval Zalo 2021 challenge. The run used AdamW, a learning rate of 2 × 10
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
