Shards, Replicas, and Cluster Architecture

Build the mental model

An index can grow too large to fit on a single machine, so Elasticsearch splits an index into smaller pieces called "shards" and distributes them across the nodes in a cluster — each shard is a complete, self-contained Lucene index. A primary shard is the one that first receives a document; a replica shard is a copy of a primary — if a node goes down, remaining replicas keep data available (redundancy), and search queries can also be distributed in parallel across replica shards, raising read throughput. This is exactly where Lesson 3's `yellow` cluster health status came from: replica shards sitting unassigned on a single-node dev environment. Shard count is fixed at index creation time (defaulting to 1 primary in recent versions); too many shards (over-sharding) raises cluster metadata overhead and wastes each node's resources (memory, file handles) unnecessarily, while too few shards misses the opportunity to distribute across all nodes in parallel, and a single overgrown shard drags query performance down. Think of library branches (nodes) sharing out books (shards) — if each branch has a duplicate copy (a replica), a fire at one branch doesn't lose the book's content, and readers can read from many branches in parallel too.

Connect it to a real scenario

The Tutorial Platform's `tutorials` index has a small data volume (hundreds of tutorials), so 1 primary shard plus 1 replica (2 shards total) is enough — pre-splitting into 10 shards for a small index like this is unnecessary overhead. Build the production cluster with 3 nodes (for data redundancy and query parallelism) — even restarting one node for maintenance leaves replica shards available on the other two, so the search service experiences zero interruption. The analytics event log index (`search-logs-*` from Lesson 16) has a larger daily volume and rolls over, so its primary shard count can reasonably be calculated higher — always treat data volume and growth rate as inputs to the shard-count decision.

• A shard is a self-contained Lucene index, used to distribute an index across nodes
• Replicas provide redundancy and parallel read throughput
• Decide shard count from data volume and growth rate — both over- and under-sharding cause problems

Try the working example

PUT /tutorials
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  }
}

GET /_cat/shards/tutorials?v

GET /_cluster/health?level=shards

You create the `tutorials` index with 1 primary and 1 replica shard, and inspect shard distribution across nodes with `_cat/shards`.

Elasticsearch Guide — Scalability and Resilience: Nodes and Shards — Elastic