An embedding is a learned map from something messy — a sentence, a paragraph, an image — to a point in a high-dimensional space where geometry means something. Points that are close together are semantically related; points far apart are unrelated. That single property is what makes semantic search (via a vector database), clustering, classification, and recommendation possible without hand-written rules. This explainer traces how that map gets built and how to use it well.

If you just need the definition, the embeddings glossary entry has it. Here we go under the hood.

From tokens to vectors

Text doesn't start as numbers, so the first step is tokenization: the model splits input into tokens — subword units like re, ##trieval, or whole common words — using a fixed vocabulary. Each token has an ID, and each ID maps to a learned row in an embedding table: a vector, initialized randomly and tuned during training. These token embeddings are the raw material, not the finished product.

The token vectors then pass through the model's layers. In a transformer-based embedding model, self-attention lets every token's representation absorb context from the others, so the vector for "bank" in "river bank" ends up different from "bank" in "savings bank." After the layers, you have one contextualized vector per token. But retrieval needs one vector for the whole passage, so the model pools them — commonly by mean-pooling the token vectors, or by reading out a special [CLS]-style token trained to summarize the sequence. That pooled vector is the embedding you store.

The crucial fact is that the whole pipeline is trained to make the geometry useful. A raw language model's hidden states are not automatically good embeddings. Dedicated embedding models are fine-tuned with a contrastive objective: show the model pairs that should be close (a question and its answer, a sentence and its paraphrase) and pairs that should be far (unrelated text), and adjust the weights until similar things land near each other and dissimilar things spread apart. The fine-tuning that produces a good embedding model is why you use text-embedding-3 or a Sentence-Transformers model rather than pulling hidden states out of a chat model.

Why the space is "semantic"

The often-quoted example — king − man + woman ≈ queen — is real for classic word embeddings and captures the intuition: directions in the space correspond to meaningful semantic relationships. Modern contextual sentence embeddings are less clean than that toy analogy, but the principle holds. During contrastive training, the only way for the model to minimize its loss across millions of pairs is to organize the space so that the axes of variation line up with the ways texts actually differ in meaning: topic, sentiment, formality, entity, intent.

Nothing labels those axes. No dimension "is" the topic dimension. Meaning is distributed across all the coordinates at once, which is exactly why you can't interpret a single embedding by reading its numbers — and why the vector is only useful in relation to other vectors. An embedding in isolation says nothing; an embedding compared to a thousand others says a great deal.

Similarity metrics: measuring closeness

Once meaning is geometry, "related" becomes "close," and you need a distance function. Three show up in practice.

Cosine similarity measures the angle between two vectors, ignoring their lengths. It's the default for text embeddings because it captures direction — semantic orientation — while discarding magnitude, which for text often reflects incidental things like length or word frequency rather than meaning. Two vectors pointing the same way score 1.0; orthogonal ones score 0; opposite ones score −1.

Dot product is cosine similarity times the two magnitudes. If your embeddings are normalized to unit length — many models output normalized vectors, or you normalize them yourself — dot product and cosine are identical, and dot product is cheaper to compute. This is why so many vector stores default to an inner-product metric on normalized vectors.

Euclidean (L2) distance measures straight-line distance. On normalized vectors it's monotonically related to cosine, so it ranks results the same way; on un-normalized vectors it behaves differently and is sensitive to magnitude.

The practical rule: use the metric the model was trained for. An embedding model optimized under cosine similarity will underperform if you query it with raw Euclidean distance on un-normalized vectors. Check the model card, and if in doubt, normalize and use cosine or inner product.

Dimensionality: bigger is not automatically better

Embeddings come in sizes — 384, 768, 1,024, 1,536, 3,072 dimensions are all common. It's tempting to read dimensionality as a quality score. It isn't.

More dimensions give the model more capacity to encode fine distinctions, which can raise retrieval quality — up to a point. But every extra dimension costs storage, memory in your vector index, and compute on every distance calculation. A 3,072-dimension embedding is 4× the memory and roughly 4× the per-comparison cost of a 768-dimension one. At a billion vectors that's the difference between a manageable bill and an unmanageable one, and it's a real input to which store you pick — pgvector vs Pinecone turns partly on how much vector data you're carrying.

Two developments matter here. First, some modern models (via a technique called Matryoshka representation learning) are trained so you can truncate the vector — keep the first 512 of 1,536 dimensions — and still get most of the quality, letting you dial the size/quality tradeoff after the fact. Second, quantization (storing dimensions as int8 or binary instead of float32) attacks the same cost from a different angle. The takeaway is that dimensionality is a tunable cost/quality knob, not a leaderboard rank. Pick the smallest embedding that hits your retrieval-quality target on your data.

Choosing and using a model

Model choice usually dominates every other decision. The differentiators that actually matter:

Domain fit. A general-purpose model trained on web text may embed legal, medical, or code text poorly because the relevant distinctions never mattered in training. Evaluate candidate models on your retrieval task, not on a public leaderboard for a different domain.

Symmetric vs asymmetric. Some models are trained for symmetric similarity (sentence ↔ sentence), others for asymmetric retrieval (short query ↔ long document) and expect you to encode queries and documents with slightly different instructions or prefixes. Using an asymmetric model symmetrically quietly tanks recall.

Match the two sides. The query and the stored documents must be embedded with the same model and version. Embeddings from different models — or even different versions of the same model — live in incompatible spaces, and comparing across them produces noise. This has a hard operational consequence: changing your embedding model means re-embedding your entire corpus. There's no incremental migration; the old and new vectors simply aren't comparable. Budget for it before you adopt a model.

Multilingual and multimodal. Multilingual embedding models place translations of the same sentence near each other, enabling cross-lingual retrieval. Multimodal models (like CLIP-style image/text models) embed images and text into one shared space, so you can search images with a text query. The same geometry-is-meaning principle powers both.

The isotropy problem and why normalization helps

A subtle failure mode worth knowing: raw embeddings from many models are anisotropic — the vectors don't spread evenly through the space but cluster into a narrow cone, so almost every pair of embeddings has a high cosine similarity regardless of meaning. When everything looks similar to everything else, your ranking loses discriminating power, and the gap between a relevant result and an irrelevant one shrinks to noise. Good embedding models are trained to counteract this, and post-processing steps — normalization, and sometimes centering or whitening the vectors — can improve separation. You usually don't need to implement these yourself, but if your retrieval returns plausible-but-wrong results with suspiciously uniform scores, anisotropy is a prime suspect, and it's a reason to prefer a model evaluated on retrieval rather than one that merely produces vectors.

This connects to a broader point: an embedding's quality is only visible relationally, through the ranking it produces on your data. You cannot inspect a vector and judge it. The only honest test is retrieval quality — does the right document rank above the wrong ones for real queries — which is why embedding-model selection is an empirical exercise, not a spec-sheet comparison. Two models with identical dimensionality can differ enormously in how cleanly they separate your domain's concepts.

Where embeddings sit in a real system

In a retrieval pipeline, embeddings are one stage of several, and their quality caps everything downstream — but they're not the whole story. A common and effective pattern is a two-stage retrieve-then-rerank: a fast embedding search pulls a broad candidate set, then a heavier cross-encoder re-scores the top candidates by looking at query and document together, which captures interactions a single pooled vector can't. Embeddings give you cheap, scalable recall; reranking gives you precision on top.

Everything you feed the embedding model also matters. Embed poorly-chunked documents — chunks that split a single idea across two vectors, or cram three ideas into one — and no metric or model recovers the lost signal. The full end-to-end version, from chunking through storage to reranking, is in how to build a RAG pipeline. Get the embedding stage right and the rest of the system has something real to work with; get it wrong and everything downstream is polishing noise.