Resource management

[maleadt]

For PkgEval, it would be nice if we could restrict resource usage of sandboxes. I'm currently doing CPU resource restriction using taskset, but it looks like limiting memory pretty much requires cgroups. That doesn't fully work in unpriviliged mode though, as we need root to create the memory cgroup, but setting joining the group seems possible without elevated permissions.

[maleadt]

Should be possible: https://www.scrivano.org/2019/02/26/resources-management-with-rootless-containers/

I can also create cgroups in systemd's cgroup2 hierarchy:

% mkdir /sys/fs/cgroup/unified/user.slice/user-23493.slice/user@23493.service/test

... but I'm not sure if systemd is going to like this, or whether I should use the debus interface.

[maleadt]

I looked some more into this, and after trying various schemes it seems like relying on systemd is the easiest (rootless) solution:

systemd-run --quiet --user --scope -p MemoryMax=2G -p CPUQuota=100% sandbox ...

Also keeps the sandbox here simple, but it won't work with Docker which requires passing --cpus and --memory etc. So if we want a truly unified interface, I guess Sandbox.jl would be the place for these options.

[staticfloat]

Yeah, I'm okay with adding some SandboxConfig options, but if systemd-run can do this, I wonder if we can't just do it manually ourselves. @Keno do you know much about this? Before I start reading up on cgroup v2 and looking at examples, I just want to see if you happen to know the theoretically optimal way of doing this.

[maleadt]

The problem with cgroup2 AFAIU is creating a cgroup where we can activate/set the resource controllers we need in an unpriviliged manner. With the old JuliaRun PR, it assumed a pre-existing cgroup with such permissions, which would complicate the use of Sandbox.jl. Furthermore, when creating a process it gets put in the current process' cgroup, and migrating it to another cgroup requires write permissions to the common ancestor, which generally isn't the case either. We could use the new clone3, which has a flag for the cgroup to join, but that's Linux 5.7+ only.

Once I realized I could use systemd-run, which communicates to the priviliged systemd daemon over DBus, I didn't look any further.

[DilumAluthge]

@maleadt Any updates here? What solution did you end up going with for PkgEval?

[maleadt]

I had something that worked, JuliaCI/PkgEval.jl@d40e757, but reverted it because Julia (i.e. libuv) doesn't support cgroup2, libuv/libuv#2323, and thus ignores any constraints and quickly runs out of memory. That's a unfortunate, since most Linux distros are moving towards cgroup2...

[maleadt]

JuliaCI/PkgEval.jl@8ea3624