Project — Tutorial Platform Search Backend

Build the mental model

This project is the capstone that pulls together everything learned across the course — mapping design (Lessons 4, 8), bulk indexing (Lessons 5, 14), `bool` queries (Lesson 10), the Node.js client (Lesson 15) — into one working system. Building a real search backend means designing three separate layers: (1) the mapping layer, explicitly deciding the `tutorials` index's field structure; (2) the sync layer, a one-time initial sync script that reads data from PostgreSQL's tutorials and lessons tables, transforms (denormalizes) it into Elasticsearch document shape, and indexes it with the `_bulk` API; (3) the query layer, an API endpoint that translates search UI requests into the Query DSL. The benefit of this layering is decoupling sync logic from query logic — the sync script can run on any schedule (cron) or be event-driven (a database trigger, change data capture), and the query layer never needs to know the underlying sync mechanism at all. When running the initial sync, tune the bulk chunk size to your PostgreSQL data volume, and create the Elasticsearch mapping before running the sync (never trust dynamic mapping) — only by running this project hands-on do all the earlier lessons' theory click into place as a production-shaped system.

Connect it to a real scenario

Create the `tutorials` index with the mapping built up across Lessons 4 and 8 (title as text plus keyword, tags as keyword, body as text with the english analyzer, publishedAt as date) — write it explicitly rather than trusting dynamic mapping. Write a Node.js sync script that queries PostgreSQL's `tutorials` joined with `lessons`, denormalizes each tutorial's lesson content into a nested array within a single document, and indexes it with the `_bulk` API (in 500-document chunks) — check each chunk's `errors` field and retry any failed items. Build the query layer (Lesson 15's `esClient`) as a search endpoint (`/api/search?q=redis&difficulty=beginner`), combining a `bool` query's `must: match(body)` with `filter: term(difficulty)`.

A search backend has a mapping layer, sync layer, and query layer, designed decoupled
Decouple the sync script from query logic — it can be scheduled or event-driven
Create the mapping before running the initial sync

Try the working example

typescript

async function syncTutorials() {
  const rows = await db.query(
    'SELECT t.id, t.title, t.published_at, t.difficulty, array_agg(l.body) AS lesson_bodies FROM tutorials t JOIN lessons l ON l.tutorial_id = t.id GROUP BY t.id'
  );

  for (let i = 0; i < rows.length; i += 500) {
    const chunk = rows.slice(i, i + 500);
    const operations = chunk.flatMap((row) => [
      { index: { _index: 'tutorials', _id: String(row.id) } },
      {
        title: row.title,
        body: row.lesson_bodies.join('\n'),
        difficulty: row.difficulty,
        publishedAt: row.published_at,
      },
    ]);
    const result = await esClient.bulk({ operations });
    if (result.errors) {
      console.error('Some documents failed to index', result.items);
    }
  }
}

You should see

All of PostgreSQL's tutorial data syncs into the Elasticsearch `tutorials` index and becomes queryable through the search endpoint.

5-minute try-it

Extend the sync script above to include an `authorName` field from PostgreSQL, add `authorName` (keyword) to the mapping, and add a filter parameter for `authorName` to the search endpoint.

One important caution

Running the sync script without creating the mapping first — dynamic mapping auto-detects, and if `difficulty` gets auto-detected as `text`, filter queries silently break.

Logging the sync script as "success" without checking the bulk chunk response's `errors` field and item-level statuses — even if some documents fail, missing data in the search index would never be noticed.

Elasticsearch Guide — Bulk API — Elastic