Sentence embeddings: SBERT

Embeddings as vectors explained what a sentence embedding is — a point in semantic space. This lesson is how you get a good one efficiently, and the answer, SBERT (Sentence-BERT), is built around a speed problem.

The cross-encoder is accurate but unscalable

The most accurate way to compare two sentences with BERT is a cross-encoder: feed both together — [sentence A] [SEP] [sentence B] — so attention runs across the pair, then output a similarity score (this is the NLI setup). It’s excellent. But it’s fatal for search: to find the best match among a million documents, you must run the full model on a million pairs, every query. The model never produces a reusable vector, so nothing can be precomputed or indexed.

The bi-encoder: encode once, compare cheaply

SBERT’s fix is the bi-encoder: pass each sentence through BERT independently to get a single fixed vector, then compare vectors with cosine similarity. Now you encode each document once, store the vectors in an index, and every query is a fast vector search:

import numpy as np

query = [1.0, 0.0]
cands = {"a dog barks": [0.9, 0.1], "the sky is blue": [0.1, 0.9], "a puppy yaps": [0.95, 0.05]}

def cos(a, b):
    a, b = np.array(a), np.array(b)
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

for c in sorted(cands, key=lambda c: cos(query, cands[c]), reverse=True):
    print(f"{cos(query, cands[c]):.2f}  {c}")

N = 1_000_000
print(f"\n{N:,} candidates: cross-encoder = {N:,} model runs/query; bi-encoder = {N+1:,} encodings + fast vector search")

1.00  a puppy yaps
0.99  a dog barks
0.11  the sky is blue
1,000,000 candidates: cross-encoder = 1,000,000 model runs/query; bi-encoder = 1,000,001 encodings + fast vector search

a puppy yaps ranks first even though it shares no words with the dog query — semantic, not lexical, matching. And the cost line is the whole point: the bi-encoder turns a per-query million-model-run problem into encode-once + cheap vector search, which is what makes semantic search and RAG feasible at scale.

How SBERT learns good vectors

A bi-encoder is only useful if the vectors actually capture meaning, and plain BERT’s vectors don’t out of the box. SBERT trains them with a siamese network: two copies of BERT with shared weights encode a pair of sentences, and a contrastive objective pulls similar sentences’ vectors together and pushes dissimilar ones apart (often using NLI data — entailment pairs as positives, contradictions as negatives, or a triplet anchor/positive/negative loss). After training, the single encoder produces vectors where cosine distance means semantic distance.

In one breath

• SBERT answers how to get good sentence vectors efficiently, around a speed problem.
• A cross-encoder (both sentences into BERT together) is accurate but unscalable — searching N documents needs N model runs per query, and it yields no reusable vector to index.
• The bi-encoder encodes each sentence independently into one vector, compared by cosine — so you encode once, index, and run fast vector search (the demo matches puppy to a dog query with no shared words).
• SBERT trains the vectors with a siamese network (shared-weight BERT) and a contrastive/triplet objective (often on NLI data), so cosine distance = semantic distance.
• Production uses both: bi-encoder retrieves top-k, cross-encoder reranks — the retrieve-and-rerank pattern behind semantic search and RAG.

Quick check

Next

These vectors are what vector databases store and what RAG retrieves; the cross-encoder shares the NLI setup. For what an embedding is geometrically, see embeddings as vectors.