Vector Databases and AI

Build the mental model

A traditional database stores application data and answers exact or relational questions. A vector database stores embeddings — numeric representations of meaning — and answers similarity questions instead.

Not substitutes for each other

The two are easy to conflate but solve genuinely different problems. In practice they are usually used together, not as replacements for each other — this pairing is exactly what powers Retrieval-Augmented Generation (RAG).

In RAG, a user's question becomes an embedding, the vector store finds the most semantically relevant text chunks, and those chunks are handed to a large language model as context so it answers grounded in real content.

A realistic AI application routes different data to different stores by kind: structured records to a relational database, raw documents to object storage, and embeddings to a vector-capable store — each doing the job it is actually good at.

Vector Database: A database or database extension optimized for storing embeddings and finding the ones most similar to a given query embedding, typically using nearest-neighbor search rather than exact matching.

Embedding: A numeric vector representation of a piece of text, image, or other data, produced by a machine learning model such that items with similar meaning end up with vectors that are close together.

text

AI APPLICATION DATA ARCHITECTURE
--------------------------------
User Data  ------------------> PostgreSQL (relational)
(accounts, orders, courses)     exact + relational queries

Documents  ------------------> Object Storage (e.g. S3)
(PDFs, uploads, raw files)      durable file storage

Embeddings ------------------> Vector Store (vector-capable DB)
(from document text)            similarity / nearest-neighbor

              PostgreSQL + Object Storage + Vector Store
                              |
                              v
                    combined as context
                              |
                              v
                        LLM ---> AI response

Connect it to a real scenario

In a realistic RAG application, a user uploading a PDF triggers three separate writes: the file to object storage, its embeddings to the vector store, and a metadata record to the relational database.

Similarity is not permission

A vector store returning a similar chunk does not mean the querying user is allowed to see it. The relational database's permission checks still have to run alongside the similarity search, not instead of it.

The code example below is intentionally small: it routes three example data types to the storage type each belongs in. For hands-on depth on embeddings and RAG pipelines, see this site's Local AI tutorial content.

Traditional databases answer exact/relational questions over application data; vector databases answer similarity questions over embeddings.
The two solve different problems and are typically used together, not as substitutes — this pairing is what powers RAG.
A realistic AI app routes data by kind: accounts to a relational database, files to object storage, embeddings to a vector store.

Try the working example

javascript

function routeDataToStorage(dataType) {
  const routingTable = {
    userAccountData: "relational database (e.g. PostgreSQL)",
    documentText: "object storage (e.g. S3-compatible bucket)",
    textEmbeddingVector: "vector store (vector-capable database)"
  };
  return routingTable[dataType] || "unknown data type";
}

for (const dataType of ["userAccountData", "documentText", "textEmbeddingVector"]) {
  console.log(dataType + " -> " + routeDataToStorage(dataType));
}

You should see

Routing the three example data types returns: userAccountData -> relational database (e.g. PostgreSQL), documentText -> object storage (e.g. S3-compatible bucket), textEmbeddingVector -> vector store (vector-capable database) — each data kind mapped to the storage type actually suited to how it will be queried.

5-minute try-it

Add a fourth case to routingTable for "chatMessageLog" (a running log of chat messages between a user and the AI assistant) and decide which of the three storage types it belongs in. Justify your choice in terms of how the data would actually be queried later.

One important caution

Trying to force similarity search into a relational database, or storing structured application data purely in a system optimized for nearest-neighbor lookups.

Trusting a vector store's similarity results as the whole answer, without a separate permission check confirming the querying user may actually see that content.

Wikipedia: Vector database — How Databases Work