Embeddings

An embedding is a list of floats — typically 256 to 3072 of them — that a neural model produces to represent the meaning of a piece of text. The model was trained so that texts with similar meaning produce numerically similar lists, which means you can then ask “how related are these two texts?” with simple arithmetic — the distance between two vectors — and no keyword matching at all. Sentences with similar meaning land near each other in that high-dimensional space; unrelated ones land far apart. Almost every “smart” feature in a modern app — search, recommendations, semantic dedup, caching, RAG — is a variation on this one idea.

What “captures meaning” actually means

Pass "the cat sat on the mat" to an embedding model and you get back something like [0.0214, -0.1031, 0.0837, ..., -0.0421] — a list of, say, 1536 floats. The individual numbers are not interpretable; what matters is that the relative distances between vectors track meaning. The vector for "the cat sat on the mat" sits close to "a feline rested on the rug" and far from "the stock market crashed". Picture each text as an arrow pointing somewhere in that space:

That is the whole of it — no magic. The model was trained to make this property hold, so now plain vector arithmetic answers questions like “which documents are most similar to this query?”

Cosine similarity

The standard distance metric for text embeddings is cosine similarity — the cosine of the angle between two vectors. Picture each vector as an arrow; cosine asks “how aligned are these two arrows?” regardless of their length. It ignores magnitude and measures only direction:

cos(a, b) = (a · b) / (||a|| × ||b||)

In numpy that is a one-liner:

def cosine(a, b):
    return (a @ b) / (np.linalg.norm(a) * np.linalg.norm(b))

The range runs from -1 (opposite) to 1 (identical direction). For real embedding models you will mostly see values between 0.3 and 0.9: below 0.5 is usually noise, and above 0.8 is usually a near-duplicate.

A tiny semantic search

Without a real embedding API we can still show the exact shape of semantic search — embed the docs, embed the query, score by cosine, sort. The toy embed below is deliberately transparent: it turns each text into a multi-hot vector over the corpus vocabulary, so the cosine is just literal word overlap and you can check the numbers by hand:

import re, math

def tokens(text):
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def embed(text, vocab):
    toks = tokens(text)
    return [1.0 if w in toks else 0.0 for w in vocab]   # one dimension per vocab word

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(x * x for x in b))
    return dot / (na * nb) if na and nb else 0.0

docs = [
    "Python is a high-level programming language.",
    "Pandas is great for tabular data analysis.",
    "Cats are popular household pets.",
    "Dogs are loyal companions and great pets.",
    "SQL is the standard language for querying relational databases.",
]
query = "How do I work with pets?"

vocab = sorted(set().union(*(tokens(d) for d in docs), tokens(query)))
index = [(d, embed(d, vocab)) for d in docs]     # embed once, query many times
q_vec = embed(query, vocab)

scored = [(cosine(q_vec, v), d) for d, v in index]
scored.sort(key=lambda x: x[0], reverse=True)

print(f"Query: {query}")
print("Ranked by cosine similarity:")
for score, doc in scored:
    print(f"  {score:.3f}  {doc}")

Query: How do I work with pets?
Ranked by cosine similarity:
  0.183  Cats are popular household pets.
  0.154  Dogs are loyal companions and great pets.
  0.000  Python is a high-level programming language.
  0.000  Pandas is great for tabular data analysis.
  0.000  SQL is the standard language for querying relational databases.

Trace one score to see there is nothing hidden. The query and the cat sentence share exactly one word, pets; the query has 6 distinct words, the cat sentence 5, so cosine is 1 / (sqrt(6) × sqrt(5)) = 1 / sqrt(30) = 0.183. The dog sentence also shares only pets but has 7 words, so it scores a touch lower at 1 / sqrt(42) = 0.154, and the rest share no words and score 0.

Be honest about what this toy is, though: it matched on literal words, so it only ranked the pet docs because the query happened to contain “pets”. A real embedding model would rank Cats are popular household pets near the query by meaning — recognising that “cats” relates to “pets” even with no shared word. That is the upgrade a real model buys; the surrounding code — embed, score, sort — does not change at all. To ship it, you would add batching, persisted vectors, metadata filters, and an approximate-nearest-neighbour index, which is exactly what a vector database provides.

Modern embedding models

A few production-ready families, as of 2026:

Choose by three things in order: quality on your domain (run MTEB-style evals on a sample of your real data), cost per million tokens, and whether you need self-hosting (regulated industries).

The dimension trade-off

More dimensions means better quality, more storage, and slower search — and the storage math is unforgiving. A 3072-dim float32 vector is about 12 KB, so 10 million documents is ~120 GB of vectors and you now need a real vector database. A 768-dim vector is about 3 KB, so the same 10 million documents is ~30 GB and fits in RAM on a beefy server. OpenAI’s text-embedding-3-large supports Matryoshka representation learning — you can truncate the vector to its first N dimensions and it still works, just slightly worse — which lets you store short 256-dim vectors for hot search and the full 3072 only when you need maximum recall.

What embeddings unlock, and when not to use them

The same vector-then-similarity pattern recurs everywhere: semantic search (rank docs by cosine to the query), recommendations (items near a liked item’s vector), deduplication (cosine > 0.95 means near-duplicate — merge or drop), semantic caching (before calling an LLM, check whether a near-identical query was answered recently and reuse it — often a 30–60% cost cut on repetitive workloads), and RAG (retrieve the top-k chunks by similarity, paste them into the prompt, and let the model answer grounded in them). Every one is “compute embeddings, store them, find nearest neighbours”; the hard parts are operational — chunking, freshness, filtering — not conceptual.

But embeddings are the wrong tool in three cases. For exact-match queries — a product SKU, an invoice number — use a regular index; embeddings will return plausible-but-wrong matches. For tiny corpora (under ~100 docs) semantic search is overkill, and a keyword search or a single LLM call with everything in context is simpler. And for adversarial input, never use raw similarity as a security boundary — an embedding happily matches the vibe of a prompt injection.

In one breath

• An embedding maps text to a vector so that similar meanings sit close together in space.
• Cosine similarity measures the angle (direction), ignoring length; normalise once and it is a fast dot product.
• Semantic search is embed → score by cosine → sort; the code is the same with a real model or a toy.
• Dimensions trade quality for storage and speed — default to 512–1024, not 3072.
• Wrong tool for exact-match IDs, tiny corpora, and as a security boundary.

Practice

What’s next

You have vectors. Now you need somewhere to store them at scale, filter on metadata, and run fast nearest-neighbour queries — a vector database, which the data-platform and Gen-AI tracks pick up from here. That also completes the Python track: from first syntax to production LLM apps, the language is now yours.