Understand distributed top-N, approximate metrics and bucket limits.
course-products examples use the teaching mapping and product from chapter 5, “Mapping and field types”. Exercise walkthroughs below use the trainer’s separate public dataset.
- Local candidates and global results
- Approximation and exhaustive traversal
- Tree cost
Understand distributed top-N, approximate metrics and bucket limits.
The course targets Elasticsearch 9.4.3. Exercise examples use the public products dataset; other requests illustrate separate teaching indices and may require setup or privileges unavailable in the trainer. Responses are shortened fragments; scores may vary. Check deployment and license requirements for advanced features.
Local candidates and global results
terms collects shard-local candidates before merging. A brand omitted locally can lose count contributions globally. size limits final output; shard_size expands local candidates at additional cost. sum_other_doc_count and doc_count_error_upper_bound help interpret missing information, but error bounds depend on bucket ordering.
Check your understandingApproximation and exhaustive traversal
cardinality estimates distinct counts; percentiles approximate distributions. Precision settings do not promise unconditional exactness. Composite aggregation pages through key combinations: use the returned after_key rather than deriving it from the final bucket. Changes between pages require a consistent read strategy.
Check your understandingTree cost
Pipeline aggregations calculate over existing buckets, such as derivatives and moving averages. They cannot recover groups removed by upstream top-N selection. Nested terms can multiply bucket counts, and high cardinality increases memory. Inspect search.max_buckets, circuit breakers and response size before scaling an analytical query.
Cardinality is an estimate; do not use it as an accounting invariant.
{
"size": 0,
"aggs": {
"brand_count": {
"cardinality": {
"field": "brand",
"precision_threshold": 1000
}
}
}
}How to read this result
Cardinality is an estimate; do not use it as an accounting invariant.
Catalog reasoning example
A brand X can rank second on each shard while only local winners are sent. Globally important X may then lack contributions. Raising final size is not always a substitute for sufficient shard_size. Use composite for exhaustive key traversal rather than indefinitely increasing terms.size. Financial reports need an explicit approximation policy or a system providing the required exact guarantees.
Check your understandingCheck your understanding
Answer before expanding each explanation.
1. Why can terms undercount?
Some local candidates were not included in shard responses.
2. What does shard_size do?
Expands local candidates at memory and transfer cost.
3. Is cardinality always exact?
No, it is approximate.
4. How do you continue composite pagination?
Pass the returned after_key.
5. Can pipelines recover omitted groups?
No, they operate on existing buckets.
Module case study: Aggregations and analytics
A shopper selects Sony but should see availability of other brands in the category. Products have multiple offers. Define each count’s scope and explain why bucket totals can differ from product counts.
Assessment criteria
Define query/post_filter or separate filter scopes, preserve access, distinguish offers from products and account for terms accuracy.
Key takeaways
- Some local candidates were not included in shard responses.
- Expands local candidates at memory and transfer cost.
- No, it is approximate.
- Pass the returned after_key.
- No, they operate on existing buckets.
Official reference
Ready to put this into practice?
Confirm reading to save your place. Reading and exercise progress are tracked separately in this browser using your guest cookie.