A vector database stores embeddings and answers one question very quickly: given this query vector, which stored vectors are closest? That nearest-neighbor search is the retrieval engine underneath RAG and semantic search.
Why it matters in production
At scale, brute-force comparison against millions of vectors is too slow, so these systems use approximate nearest-neighbor indexes that trade a little recall for a lot of speed. The database also handles metadata filtering — "only search documents this user can see" — which turns out to be where a lot of real-world correctness lives.
The common mistake
Buying a dedicated vector database too early. If you already run Postgres, pgvector often carries a small-to-medium corpus fine, and you have deleted an entire system from your architecture instead of adding one that must be provisioned, backed up, and kept in sync with your source of truth. We wrote the tradeoff up in detail: pgvector vs Pinecone.
How we actually use it
We start with the simplest store that works and only graduate to a dedicated engine when index build time, memory pressure, or write churn shows up in metrics. When the corpus does justify it, the choice between the popular managed and self-hosted options matters less than people think — see Pinecone vs Weaviate vs Qdrant. Whatever the store, a reranking pass on top usually buys more accuracy than switching databases.