set MALLOC_ARENA_MAX to a lower dynamic value to prevent OOM #143
Conversation
The default behavior is 8x the number of detected CPUs . As Cloud Foundry typically uses large host machines with smaller containers, and the Java process is unaware of the difference in allocated CPUs, the numbers are way off. This often leads to high native memory usage, followed by a cgroup OOM killer event. We go with Heroku's recommendation of lowering to a setting of 2 for small instances. We also grown the setting linearly with memory to be more in line with the default setting in Mendix Cloud v3. References: - cloudfoundry/java-buildpack#163 - https://devcenter.heroku.com/articles/testing-cedar-14-memory-use - cloudfoundry/java-buildpack#320
There was a problem hiding this comment.
It seems wrong to make this value memory dependent. It should be dependent on the number of cores available for your process (give each core's threads their own set of memory pools).
I cannot see the "why" to make this memory dependent. ("This is how we do it in V3" does not count.) Btw, I could live with a static value of 2 or 4, especially since we can tweak it at runtime.
|
@hansthen in CF, the amount of CPU shares (not cores) scales linearly with the amount of memory. See also CLD-2455. I would also be fine with hard-coding to 2 for now. |
|
Since we don't have any real life measurements of actual usage of these memory areas (as opposed to allocation of them) and/or malloc/free and lock contention rates in hosted applications, it makes sense to do anything to at least take the first step to lower the limits to sensible levels that correspond to cpu/memory sizing of the container limits, compared with running the same in a normal OS. |
|
What @knorrie said, +1 that. |
jtwaleson
dismissed
hansthen’s stale review
discussed offline, we're good with the current proposal, but as the change to 2 might be too limited for some apps, we'll use the dynamic calculation.
|
discussed offline, we agreed on the current calculation |
The default behavior is 8x the number of detected CPUs . As Cloud
Foundry typically uses large host machines with smaller containers,
and the Java process is unaware of the difference in allocated CPUs,
the numbers are way off. This often leads to high native memory usage,
followed by a cgroup OOM killer event.
We go with Heroku's recommendation of lowering to a
setting of 2 for small instances. We also grown the setting linearly
with memory to be more in line with the default setting in Mendix
Cloud v3.
References: