Support page size > 4K in jemalloc

[koraykoska]

Description

Some ARM processors have a page size other than 4k (16k for example). lighthouse bn is unable to launch on those processors, with the error: <jemalloc>: Unsupported system page size

This is a know issue as can be seen here:

jemalloc/jemalloc#2572

As explained in the above issue, jemalloc page size is a compile time option, but higher ones are ok to use on lower page sizes, the other way round doesn't work.

My suggestion would be to increase the page size to 16k during the jemalloc compilation phase on the ARM docker images. As it's officially supported to have it higher, this shouldn't break anything, just make 16k systems work.

Version

v4.6.0

Present Behaviour

On launch of the BN: : Unsupported system page size

Expected Behaviour

Just work normally. Page size shouldn't be an issue for normal applications.

Steps to resolve

Right now the only way is to switch your kernel to a 4k page size. That's quite intrusive.

[michaelsproul]

I think a better option might be a feature flag that compiles jemalloc with 16K pages, which we can then decide to turn on or off by default.

I'm hesitant to turn it on by default without assessing the performance impact

[koraykoska]

@michaelsproul What about adding it by default to ARM builds? It's more common than 4k on ARM

[michaelsproul]

@koraykoska Yeah I think that would probably be fine, but we should do some benchmarking first. Let's start with an optional feature, then we can switch it on for the ARM builds by default if it proves useful.

Btw, if you build from source on the board, I think jemalloc will already autodetect the page size, won't it? So this issue is more about cross-compiling, which includes the case of building the release binary, right?

[koraykoska]

@michaelsproul Yes it's about the cross-compilation and especially also the docker image.

I would be happy with a special docker tag like arm-16k or something.

[michaelsproul]

Sounds good. I'll see if anyone from the team is interested in working on it

[jtraglia]

@koraykoska @michaelsproul we ran into the same issue with Reth. This was our fix:

• fix: tell jemalloc to use 64-KiB pages when cross building aarch64 paradigmxyz/reth#7123

[bcaddy]

I think I'm also having this issue. I'm trying to use Ruff on a Grace Hopper and getting this error

<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
memory allocation of 5 bytes failed
Aborted

Any suggestions on how to fix it? I installed Ruff with Conda from the Conda-forge channel.

[ph03]

Pushed a PR (#6831) addressing this issue (in a similar way to paradigmxyz/reth#7123) - tested this both on a ARM raspberry-pi 4 kernel using 64-KiB pages (using a custom kernel build for it's BCM2711 chip), as well as on an official 16-KiB page-size ARM raspberry-pi 5 kernel (see tracking issue raspberrypi/bookworm-feedback#107)

[jimmygchen]

Completed in #6831 🎉