Linux Jails

This is intended as a revision to https://wiki.freebsd.org/LinuxJails

This page describes the steps required to install and configure a complete Linux (Ubuntu) user space environment within a FreeBSD jail, using Linuxulator and debootstrap.

Required steps

1. Run FreeBSD 13.0-RELEASE or newer on amd64, or 14-CURRENT on arm64
2. Enable linux(4) compatibility, as documented in the man page; this boils down to running sysrc linux_enable=YES; service linux start, as root. That loads the necessary kernel modules and sets some sysctls.
3. Create a FreeBSD jail in the usual way, with expanded mount capabilities, which are only needed during installation:

jail -c \
    name=ubuntu \
    host.hostname="ubuntu.example.com" \
    path="/jails/ubuntu" \
    interface="lo1" \
    ip4.addr="10.11.12.13" \
    ip6.addr="2001:db8:::/32" \
    exec.start="/bin/sh /etc/rc" \
    exec.stop="/bin/sh /etc/rc.shutdown" \
    mount.devfs \
    devfs_ruleset=5 \
    allow.mount \
    allow.mount.devfs \
    allow.mount.fdescfs \
    allow.mount.procfs \
    allow.mount.linprocfs \
    allow.mount.linsysfs \
    allow.mount.tmpfs \
    enforce_statfs=1

The debootstrap script performs several mounts (varies by distro & version) as it configures the chroot environment in /compat/linux, hence the need for all the allow.mount* additions. Also, the devfs declarations aren't redundant. mount.devfs will mount /dev within the jail , whereas allow.mount.devfs will allow debootstrap to mount a devfs at /compat/linux/dev.

4. Within the jail:

   a. install sysutils/debootstrap with pkg install debootstrap.

   b. run debootstrap jammy /compat/linux. This will install Ubuntu Jammy into /compat/linux. If the console output doesn't end with "Base system installed successfully", then check the debootstap log at /compat/linux/debootstrap/debootstrap.log for errors and clues for what went wrong. (Reading the logs and tracking the errors back to the install scripts to see what fails is how I was able to get Ubuntu successfully installed within a jail).

5. Shut down the jail with jail -r ubuntu

6. Add the fs mounts in the jails config file, either /etc/jail.conf or /etc/jail.conf.d/$name.conf. To share home directory contents and run X11 apps, you will also need null mounts for /home and /tmp. Here's a working /etc/jail.conf.d/ubuntu.conf:

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
devfs_ruleset=5;
path = "/jails/$name";
interface = lo1;
host.hostname = $name;

ubuntu	{
		ip4.addr = lo1|10.11.12.13;
		ip6.addr = lo1|2001:db8:::/32;
		mount += "devfs     $path/compat/linux/dev     devfs     rw  0 0";
		mount += "tmpfs     $path/compat/linux/dev/shm tmpfs     rw,size=1g,mode=1777  0 0";
		mount += "fdescfs   $path/compat/linux/dev/fd  fdescfs   rw,linrdlnk 0 0";
		mount += "linprocfs $path/compat/linux/proc    linprocfs rw  0 0";
		mount += "linsysfs  $path/compat/linux/sys     linsysfs  rw  0 0";
		mount += "/tmp      $path/compat/linux/tmp     nullfs    rw  0 0";
		mount += "/home     $path/compat/linux/home    nullfs    rw  0 0";
	}

7. Start the jail in the usual way: service jail start ubuntu

While the jail is running, you can run jexec ubuntu chroot /compat/linux /bin/bash and use apt like you would on a typical Ubuntu instance. You will probably want to add users and groups with UIDs/GIDs matching the ones in the jail. While systemd doesn't work, the service command works as usual. Some things that would probably get set up by the installer require manual tweaking: if you get warnings about LC_ALL you'll want to install appropriate locales (apt search language-pack). To get your environment variables set up in the typical Ubuntu way you might want to start sshd(8) (service ssh start) and log in this way; you can add chroot /compat/ubuntu /usr/sbin/service ssh start in the jails /etc/rc.local to make it run at boot. As with any other Ubuntu, you will probably want to add the package repositories missing from defaults; for amd64 (x86_64) version of Jammy make your /compat/ubuntu/etc/apt/sources.list look like this:

deb http://archive.ubuntu.com/ubuntu jammy main universe restricted multiverse
deb http://security.ubuntu.com/ubuntu/ jammy-security universe multiverse restricted main
deb http://archive.ubuntu.com/ubuntu jammy-backports universe multiverse restricted main
deb http://archive.ubuntu.com/ubuntu jammy-updates universe multiverse restricted main

For arm64, it should look like this:

deb http://ports.ubuntu.com/ubuntu-ports jammy main universe restricted multiverse

You can find the list of tested apps at LinuxApps. Feel free to add new entries.

How is this Different?

I found a number of HowTo articles on "FreeBSD Linux jails" but all of them (including the canonical one) require installing debootstrap or linux_base-c7 on the host. That does work, and it is easier, but I want all the linux related stuff within a jail. Especially since debootstrap has 27 dependencies. I prefer to keep the host OS succinct and push deps down into the jails that need them.