Thuta Learning
IntermediateData & Databasesbeginner

Relevance Scoring (BM25) and Highlighting

What you'll walk away with

  • Explain the core ideas behind Relevance Scoring (BM25) and Highlighting
  • Run the sample Elasticsearch query or code and verify its output
  • Apply the technique correctly to the Tutorial Platform and production scenarios

Build the mental model

To compute `_score`, Elasticsearch uses the BM25 ranking algorithm by default, which combines three factors: term frequency (how many times the search term repeats in a document — more repeats raise the score, with diminishing returns), inverse document frequency (how rare a word is across the whole collection — a rarer term is more discriminating across the document set, so it raises the score more), and field length normalization (compensating so a match in a short field counts for more than the same match in a long field). This means if "redis" appears in nearly every document (say, an entire Redis-focused course), its discriminating power drops and its score contribution shrinks. Highlighting is a related feature — it wraps a matched term, along with surrounding context from the document body, in a `<em>` tag (customizable) and returns it as a snippet, so a user can instantly see why a result is relevant without reading the whole document. This directly parallels the bolded text snippets in a Google search result list ("...this page mentions the term **redis cache**...") — a UX pattern that lets a user's eye scan each result's relevance instantly. Enabling highlighting in production adds extra computation cost, so it's better applied selectively to user-visible search result pages rather than turned on automatically for every result list.

Connect it to a real scenario

On the Tutorial Platform's public search results page, when a user searches "session store", attach a `highlight` block to the `body` field and render the snippet from `highlight.body` in the response ("...uses Redis as a **session store**...") inside the result card — this makes clear why a lesson is relevant far better than a title-only preview. In an admin search-quality debugging tool, add the `explain: true` parameter to see `_score` broken down by each BM25 factor — when result ranking looks wrong, use that breakdown as a tracing tool.

Try the working example

http
GET /tutorials/_search
{
  "query": {
    "match": { "body": "session store" }
  },
  "highlight": {
    "fields": {
      "body": { "fragment_size": 150, "number_of_fragments": 1 }
    }
  }
}
You should see
Each search result document includes a `highlight.body` array with a snippet wrapping the matched terms in `<em>` tags.

5-minute try-it

Write a query searching "elasticsearch" on the `title` field, add a highlight block for `title`, and also add `explain: true` to see the `_score` breakdown.

One important caution

Treating `_score` as an absolute, comparable value and directly comparing scores from two structurally different queries — BM25 scores are only meaningful relative to their own query's context, so cross-query comparison is misleading.

Leaving highlighting on by default for every result list, running it over a large `body` field — the extra computation to find matched snippets adds unnecessary latency to every request.

Elasticsearch Guide — HighlightingElastic

Easy traps

  • Treating `_score` as an absolute, comparable value and directly comparing scores from two structurally different queries — BM25 scores are only meaningful relative to their own query's context, so cross-query comparison is misleading.
  • Leaving highlighting on by default for every result list, running it over a large `body` field — the extra computation to find matched snippets adds unnecessary latency to every request.
  • Validate sample queries and requests on a local or test instance with recoverable data before applying them to production.

Exercise

Write a query searching "elasticsearch" on the `title` field, add a highlight block for `title`, and also add `explain: true` to see the `_score` breakdown.

You'll know it worked when: Each search result document includes a `highlight.body` array with a snippet wrapping the matched terms in `<em>` tags.

Relevance Scoring (BM25) and Highlighting | Thuta Learning