Multilingual Semantic Search
What You'll Learn
You'll learn how to use multilingual sentence transformers to enable cross-lingual retrieval, where queries and documents can be in different languages.
The Core Idea
Imagine searching a library where books are written in dozens of languages, but you can ask questions in your native tongue and still find the right book—even if it's written in a completely different language. That's multilingual semantic search.
Multilingual sentence transformers extend the bi-encoder architecture you've learned to work across languages. Instead of training separate models for each language pair, these models map sentences from all languages into a shared embedding space. A question in English and its answer in Spanish will have similar vector representations if they convey the same meaning.
How It Works
The key is training on parallel and comparable data across languages. During training, the model learns that "Hello, how are you?" (English) and "Hola, ¿cómo estás?" (Spanish) should produce embeddings that are close together in vector space, while unrelated sentences should be far apart—regardless of language.
At search time, you encode your query in one language and compare it against document embeddings in any other language using the same similarity metrics (like cosine similarity) you'd use for monolingual search. The shared embedding space acts as a universal translator for semantic meaning.
This approach handles asymmetric search tasks naturally: short queries in French can retrieve long documents in German, Japanese, or Arabic without needing explicit translation steps or language detection.
Popular multilingual models include variations of sentence transformers pretrained on 50+ languages, often using contrastive learning objectives with multilingual training data.
Key Takeaway: Multilingual sentence transformers enable cross-lingual semantic search by mapping all languages into a shared embedding space, allowing queries and documents in different languages to be directly compared without translation.