---
title: "What is a vector database?"
excerpt: "How vector databases work: ANN indexes (HNSW, IVF, PQ), the recall–latency–memory tradeoff, and when you need one versus pgvector."
date: "2026-07-09"
lastModified: "2026-07-09"
summary: "A vector database stores high-dimensional embeddings and answers nearest-neighbor queries in sub-linear time using approximate indexes. This explainer covers how HNSW, IVF, and product quantization work, the tradeoffs between them, and the honest question of whether you need a dedicated vector store at all."
keyTakeaways:
  - "A vector database is a nearest-neighbor engine: it trades exact search for approximate search to make similarity queries fast at scale."
  - "HNSW gives the best recall-at-latency but costs RAM; IVF partitions the space; product quantization compresses vectors at the cost of precision."
  - "Below a few million vectors, pgvector on Postgres is usually the right answer — a dedicated vector database earns its keep at scale or with heavy metadata filtering."
author: "Teo Deleanu"
authorAvatar: "/team/teo.jpg"
tags: ["Vector Database", "Embeddings", "RAG", "AI Engineering"]
keywords:
  - "what is a vector database"
  - "how vector databases work"
  - "hnsw index"
  - "approximate nearest neighbor"
  - "ivf pq quantization"
  - "pgvector vs vector database"
---

A vector database is a system built to answer one question fast: given this vector, which of the millions I've stored are closest to it? That sounds narrow, and it is — but it's the operation that powers semantic search, [retrieval-augmented generation](/glossary/retrieval-augmented-generation), recommendation, deduplication, and clustering. The whole category exists because doing that operation exactly, at scale, is too slow, and doing it approximately is a genuinely hard engineering problem.

This explainer walks through what's actually inside a vector database: the geometry of the search problem, the approximate-nearest-neighbor (ANN) indexes that make it tractable, and the tradeoffs that decide which index — and which product — you should reach for. If you want the one-paragraph version first, the [vector database glossary entry](/glossary/vector-database) has it.

## The problem: nearest neighbors in high-dimensional space

Start with the data. An [embedding model](/glossary/embeddings) turns a chunk of text, an image, or an audio clip into a fixed-length list of floating-point numbers — a vector. A typical text embedding has 384, 768, or 1,536 dimensions. Semantically similar inputs land near each other in this space; unrelated ones land far apart. "Nearness" is measured by a distance metric, usually cosine similarity or Euclidean (L2) distance.

Search, then, is geometry. You embed the query, and you want the *k* stored vectors nearest to it. The brute-force approach — compute the distance from the query to every stored vector and keep the smallest *k* — is called a flat or exact search. It's perfectly accurate and completely correct. It's also O(n · d): for a million vectors at 768 dimensions, that's ~768 million multiply-adds *per query*. At a few thousand vectors this is fine and you should just do it. At tens of millions, with a latency budget of milliseconds and thousands of queries per second, it collapses.

The curse of dimensionality makes it worse. In high dimensions, the classic spatial data structures that give logarithmic search in 2D or 3D — k-d trees, R-trees — degrade to scanning nearly everything. Distances between points also concentrate: the nearest and farthest neighbors become almost equidistant, so any structure that relies on tight partitioning struggles. This is why vector databases don't use tree structures from the GIS world. They use approximate methods designed for hundreds of dimensions.

## The core trade: exact for approximate

The central bet of every vector database is that you'll accept *approximate* answers in exchange for speed. Instead of guaranteeing the true top-*k*, an ANN index returns *most* of the true top-*k* *most* of the time, and does it in sub-linear time. The quality of that approximation is **recall**: if the true 10 nearest neighbors are set T and the index returns set R, recall@10 is |R ∩ T| / 10. A recall of 0.95 means, on average, you get 9.5 of the 10 truly-closest results.

Everything downstream is a knob on the recall–latency–memory triangle. You can almost always buy more recall by spending more time or more RAM, and every index exposes parameters that move you along that curve. The art of running a vector database is picking the operating point your application actually needs — and most RAG applications need far less than 0.99 recall, because the generator tolerates a slightly imperfect context set.

## HNSW: navigable small-world graphs

The dominant index today is HNSW — Hierarchical Navigable Small World. It's the default in Qdrant, Weaviate, Milvus, and pgvector's newer index type, and it usually wins the recall-at-a-given-latency benchmark.

The intuition is a multi-layer graph. Every vector is a node. Nodes are connected to their neighbors by edges, forming a "navigable small-world" graph where, like a social network, any two nodes are reachable in a few hops. HNSW stacks several such graphs in layers: the top layer is sparse (few nodes, long-range links), and each layer down is denser, until the bottom layer contains every vector.

A search enters at the top layer at an arbitrary entry point and greedily walks toward the query — at each node, it hops to whichever neighbor is closer to the query vector. When it can't get closer at the current layer, it drops down a layer and continues. The top layers act like an express train covering long distances in few hops; the bottom layer does the fine-grained local search. The result is roughly logarithmic hops instead of a linear scan.

Two parameters dominate. `M` is the number of edges per node — higher `M` means a richer graph, better recall, and more memory. `efConstruction` controls how hard the index works to find good neighbors when inserting; `efSearch` (or just `ef`) controls how wide the search beam is at query time. Raising `efSearch` explores more of the graph, lifting recall at the cost of latency — this is your main runtime recall knob.

HNSW's weakness is memory and mutability. The graph lives in RAM for speed, and the edges themselves add substantial overhead on top of the raw vectors. Deletes are awkward — most implementations tombstone a node and only truly reclaim it on a rebuild — so a high-churn dataset gradually degrades until you re-index.

## IVF: inverted file partitioning

The other major family is IVF — Inverted File index. Where HNSW builds a graph, IVF partitions the space. During a training step, it runs k-means clustering over a sample of your vectors to find `nlist` centroids — think of them as buckets. Every vector is assigned to its nearest centroid's bucket.

At query time, you don't scan every bucket. You compute the query's distance to each centroid, pick the `nprobe` closest buckets, and search only within those. If `nlist` is 4,096 and `nprobe` is 16, you've cut the search space to roughly 16/4,096 — under half a percent — of the vectors. `nprobe` is the recall knob: probe more buckets, get closer to exact, spend more time.

IVF trains faster than HNSW builds and uses less memory per vector, which makes it attractive at very large scale. Its weakness is the partition boundary problem: a true nearest neighbor sitting just across a cluster boundary in an un-probed bucket gets missed. Getting high recall on adversarial query distributions can force `nprobe` up until the speed advantage shrinks. IVF also needs enough data to train meaningful centroids — it's a poor fit for a small or rapidly-changing collection.

## Product quantization: compressing the vectors

Both HNSW and IVF still store full-precision vectors, and at a billion vectors those floats dominate your memory bill. Product quantization (PQ) attacks that directly by *compressing* the vectors themselves.

PQ splits each vector into *m* sub-vectors, then runs k-means on each sub-space to learn a small codebook (typically 256 centroids, so one byte per sub-vector). A stored vector becomes a list of *m* codebook indices instead of *d* floats. A 768-dimensional float32 vector — 3,072 bytes — split into 96 sub-vectors of one byte each becomes 96 bytes, a 32× compression. Distances are then estimated from precomputed codebook lookup tables rather than the original floats.

The cost is precision: PQ distances are approximate approximations, so recall drops. In practice PQ is layered under a coarse index (the common `IVF-PQ` combination) and paired with a re-ranking step — retrieve a larger candidate set using the compressed vectors, then re-score the top candidates against full-precision vectors kept on disk. This gives you most of the memory savings while clawing back most of the recall. It's the standard recipe for billion-scale search.

## What a vector database adds around the index

An index is not a database. The reason you reach for Pinecone, Qdrant, Weaviate, or Milvus rather than a raw library like FAISS is everything wrapped around the index: persistence and crash recovery, horizontal sharding and replication, real-time inserts and deletes without a full rebuild, and — the feature people underrate — **metadata filtering**.

Filtering is where architectures diverge sharply. Say you want the nearest vectors *where `tenant_id = 42` and `published = true`*. Pre-filtering (restrict to matching rows, then search) is exact but can be slow if the filter is unselective. Post-filtering (search first, then drop non-matching results) is fast but can return fewer than *k* results, or none, when the filter is selective. Good vector databases integrate the filter *into* the graph or bucket traversal so it stays both fast and correct — and how well they do this is a real differentiator between products. If you're choosing between managed and self-hosted options, [pgvector vs Pinecone](/compare/pgvector-vs-pinecone) and [Pinecone vs Weaviate vs Qdrant](/compare/pinecone-vs-weaviate-vs-qdrant) dig into exactly these tradeoffs.

## Do you actually need one?

Here's the honest part. A dedicated vector database is often the wrong first move.

If you're already on Postgres and have fewer than a few million vectors, **pgvector** is very likely the right answer. It gives you HNSW and IVFFlat indexes inside the database you already operate, backed up, transactional, and — crucially — able to filter on your existing SQL columns and JOIN vectors against real relational data in one query. You avoid running, securing, and syncing a second datastore. Most early RAG systems never outgrow it.

You start earning a dedicated vector database when one of these becomes true: you're past the tens-of-millions of vectors where a single Postgres box strains; you need sub-10ms latency at high QPS that a general-purpose database can't hit; your write volume and churn demand real-time index maintenance; or your filtering is complex enough that a purpose-built hybrid query engine measurably beats a bolt-on index. Scale and filtering complexity, not the mere presence of embeddings, are the signals.

The failure mode to avoid is adopting a distributed vector database on day one "because we're doing AI," then operating a sharded, replicated system to serve fifty thousand vectors that would fit comfortably in a Postgres table — or in memory. Start with the simplest thing that meets your recall and latency targets, measure, and move up the ladder only when the numbers say you must. When you do build the retrieval layer, [how to build a RAG pipeline](/guides/how-to-build-a-rag-pipeline) covers wiring the store into an end-to-end system, and getting your [chunking](/glossary/chunking) right matters more for retrieval quality than which index you picked.