Embeddings and Vector Databases

Build the mental model

A chat model like Llama or Mistral is trained to do one job: given some text, predict the next likely word. An embedding model is trained for a completely different job: given some text, output a fixed-length list of numbers, called a vector, that represents that text's meaning in a way a computer can compare mathematically.

This matters because it means an embedding model is not a smaller or simpler chat model, it is a different kind of model entirely, usually with its own separate API endpoint, such as /api/embeddings in Ollama, even though it might live in the same local server.

The key property of a good embedding is that texts with similar meaning end up with similar vectors, and texts with unrelated meaning end up with dissimilar vectors, even if they don't share any of the same words. “The cat is sleeping” and “a kitten is napping” would land close together in vector space, while “the cat is sleeping” and “quarterly revenue increased” would land far apart.

This closeness is measured with a similarity score, most commonly cosine similarity, which looks only at the angle between two vectors, ignoring their length.

Once you can turn any piece of text into a vector and measure how close two vectors are, you can build a search engine based on meaning instead of exact keywords, which is exactly what a vector database like Chroma or Qdrant is built for: it stores many vectors alongside their original text, and given a new query vector, quickly finds the top-K stored vectors most similar to it.

Embedding

The fixed-length numeric vector an embedding model produces from a piece of text, positioned so that texts with similar meaning end up with similar vectors.

Vector

A fixed-length list of numbers — in this context, the numeric representation of a piece of text's meaning that a computer can compare mathematically.

Vector Database

A database purpose-built to store many vectors alongside their original text or data, and given a new query vector, quickly find the top-K most similar stored vectors.

TEXT TO VECTOR TO TOP-K RESULTS
-------------------------------
------------------------------------------------------------
  TEXT
  "the cat is sleeping"
     |
     |  embedding model
     v
  VECTOR
  [0.91, 0.08, 0.02, 0.41, ...]
     |
     |  compare against many stored vectors
     |  using cosine similarity
     v
  SIMILARITY SEARCH
  score("cat is sleeping", "kitten is napping")   = 0.99  (close)
  score("cat is sleeping", "quarterly revenue")   = 0.15  (far)
     |
     v
  TOP-K RESULTS  (the K stored texts with the highest scores)

Connect it to a real scenario

Suppose you're building a “search my notes by meaning” feature for a personal knowledge base of a few hundred short notes.

The code exercise below has you implement that comparison step yourself with plain Python and no external library, using small hardcoded vectors that stand in for real embedding output, specifically to make the mechanics concrete before you ever call a real embedding model.

Once you see that “similar meaning” really does reduce to “a similarity score between two lists of numbers,” swapping in a real embedding model and a real vector database later is a matter of plumbing, not new concepts, since the underlying math you just implemented by hand is the same math those libraries run for you.

• An embedding model turns text into a fixed-length vector; a chat model predicts the next word — they are different kinds of models entirely.
• Cosine similarity measures how close two vectors are by angle, and is the standard way to score semantic similarity.
• A vector database exists to store many vectors and quickly return the top-K most similar ones to a query.

Try the working example

import math


def cosine_similarity(vector_a, vector_b):
    dot_product = sum(a * b for a, b in zip(vector_a, vector_b))
    magnitude_a = math.sqrt(sum(a * a for a in vector_a))
    magnitude_b = math.sqrt(sum(b * b for b in vector_b))
    return dot_product / (magnitude_a * magnitude_b)


if __name__ == "__main__":
    # Toy 4-dimensional "embeddings" standing in for real model output.
    cat = [0.9, 0.1, 0.0, 0.4]
    kitten = [0.85, 0.15, 0.05, 0.35]
    bicycle = [0.05, 0.9, 0.8, 0.1]

    pairs = [
        ("cat", "kitten", cat, kitten),
        ("cat", "bicycle", cat, bicycle),
        ("kitten", "bicycle", kitten, bicycle),
    ]

    for name_a, name_b, vec_a, vec_b in pairs:
        score = cosine_similarity(vec_a, vec_b)
        print(f"similarity({name_a}, {name_b}) = {score:.4f}")

Running the script prints three similarity scores, one per pair: `similarity(cat, kitten) = 0.9964`, `similarity(cat, bicycle) = 0.1462`, `similarity(kitten, bicycle) = 0.2238`. The 'cat' and 'kitten' vectors, built to resemble each other, score close to 1.0 (highly similar), while 'bicycle', built to differ, scores much lower against both.

Chroma documentation — Local AI / Local LLM