Elasticsearch 7: “Data too large”#
Since release 2.9.0, EclecticIQ Intelligence Center comes bundled with
Elasticsearch (ES) 7.9.1. ES 7 adds a new
real memory circuit breaker
that causes ES nodes to respond with a
circuit_breaking_exception
error when it detects that
memory use has reached 95% of the totally available JVM
heap.
Because of this change, you may encounter issues related to available memory where previously at the same workloads, ES would appear to run smoothly.
If your plaform is encountering issues related to
Elasticsearch responding with a
circuit_breaking_exception
error, you can do the
following to mitigate:
Increase available memory for ES#
The circuit_breaking_exception
error occurs only when ES
detects that you are about to go over a memory use threshold
that would cause it to fail.
Increase the amount of memory available to ES, or move it to its own host where it does not compete with EclecticIQ Intelligence Center for resources to keep your ES nodes running.
(Not recommended) Disable the “real memory circuit breaker”#
Warning
This may allow ES to reach an out of memory state and fail.
(Not recommended) To disable the “real memory circuit breaker”, set the
indices.breaker.total.use_real_memory
parameter in your ES
configuration to false
.
This allows ES to use the ES 6 parent circuit breaker instead, but disables the safety guarantees that the real memory circuit breaker provides.