Follow a write from the primary shard to search visibility and durable storage.
- Primary + replicas
- Buffer and translog
- Refresh → search
Follow a write from the primary shard to search visibility and durable storage.
The course targets Elasticsearch 9.4.3. Exercise examples use the public products dataset; other requests illustrate separate teaching indices and may require setup or privileges unavailable in the trainer. Responses are shortened fragments; scores may vary. Check deployment and license requirements for advanced features.
Shards and routing
An index is partitioned into primary shards; replicas copy those shards. Routing chooses a primary, normally using a hash derived from the ID. A coordinating node fans out searches and merges local results. More shards do not automatically make searches faster: coordination and per-shard overhead increase.
Check your understandingRefresh, flush and translog
New operations enter buffers and the translog. Refresh opens searchable segments but is not a full durable Lucene commit. Flush creates a commit and starts a new translog generation. Write acknowledgment and search visibility are separate events. refresh=wait_for supports read-after-write search at the cost of waiting.
Check your understandingSegments and background work
Segments are mostly immutable: updates create a new version and mark the old one deleted. Merges consolidate segments and reclaim deleted data, consuming CPU and disk I/O. Force-merging an actively written index is not routine tuning; first understand the workload and data lifecycle.
Inspect shard health alongside write latency; green is not a relevance metric.
GET /_cluster/healthHow to read this result
Inspect shard health alongside write latency; green is not a relevance metric.
Catalog reasoning example
During a million-document rebuild, each operation rarely needs immediate search visibility. Batching and a suitable refresh interval reduce tiny segments. Editing one product and immediately searching for it is a different freshness requirement, where waiting for refresh may be reasonable. Classify write flows first. Replicas can improve read capacity and resilience, but they also receive writes and do not eliminate indexing cost.
Check your understandingCheck your understanding
Answer before expanding each explanation.
1. What does routing select?
The document primary shard; custom routing must be used consistently.
2. Does refresh provide a backup?
No, it makes segments searchable.
3. Why a translog?
To recover acknowledged operations not yet in a Lucene commit, under the durability policy.
4. Why do updates consume resources?
A new version is written, the old one is logically deleted, and merges reclaim space.
5. What does refresh=wait_for do?
Waits for search visibility instead of unconditionally forcing a refresh.
Module case study: How Elasticsearch works
A product is saved and indexed but an immediate search misses it. The team proposes more replicas. Explain write acknowledgment, refresh and why replicas do not solve search visibility.
Assessment criteria
Separate acknowledgment from visibility, consider refresh=wait_for for the relevant flow, and retain transactional validation in the primary database.
Key takeaways
- The document primary shard; custom routing must be used consistently.
- No, it makes segments searchable.
- To recover acknowledged operations not yet in a Lucene commit, under the durability policy.
- A new version is written, the old one is logically deleted, and merges reclaim space.
- Waits for search visibility instead of unconditionally forcing a refresh.
Official reference
Ready to put this into practice?
Confirm reading to save your place. Reading and exercise progress are tracked separately in this browser using your guest cookie.