NetBSD / OpenBSD support, take 2

[anarazel]

Hi,

#311 was closed after #794 was done. To quote from #311 "The only thing that is missing is OpenBSD images for Google Compute Engine like FreeBSD has". A colleague is working on automating net/openbsd image generation.

I assume that from the cirrus side we already could utilize those images by starting them as persistent workers? Provided we compile the agent from source successfully?

What would be necessary to support starting VMs on-demand in GCP (community or non-community cluster)? I assume the sticking point would be a) prebuilt agent for $OS b) a way for cirrus to start the agent? Currently the gcloud services don't run on open/netbsd. Is the agent started via ssh or from a startup script?

Regards,

Andres

[anarazel]

FWIW, @nbyavuz got the image generation working, including ssh and startup scripts. I think he plans to open an issue for making the agent work on net/openbsd.

[fkorotkov]

@anarazel awesome news! Once there is an agent binary ready for openbsd it will be pretty fast to enable it. I think right now the agent build is failing for OpenBSD because of mitchellh/go-ps#45 which hopefully will be merged soon.

[nbyavuz]

Hi,

I am able to use openBSD / netBSD as a persistent worker now(see) but I can't use them as compute engine instances. I am using nbyavuz/openbsd-netbsd-packer repository for creating openBSD / netBSD images. When I use compute engine instances, tasks fail with Agent is not responding error(see).

Example repository for running images as persistent workers and compute engine instances: nbyavuz/openbsd-netbsd-cirrus

I have a startup script on these images to download ssh-keys and make them ready for use(here).
Important part:

instance_keys=$(/usr/local/bin/curl -s -H "Metadata-Flavor: Google"  http://metadata.google.internal/computeMetadata/v1/project/attributes/ssh-keys)

if [ "$instance_keys" != "" ]
then
    echo "$instance_keys" | while read line
    do
        username="$(echo $line | cut -d: -f1)"
        user_key="$(echo $line | cut -d: -f2-)"
        key_comment="$(echo $line | awk '{print $NF}')"

        if [ "$username" != "" ]
        then
            /usr/sbin/useradd ${username}
            mkdir -p /home/${username}/.ssh
            touch /home/${username}/.ssh/authorized_keys
            chown -R ${username}:${username} /home/${username}

            if [ "$(grep -c "$user_key" /home/${username}/.ssh/authorized_keys)" == "0" ]
            then
                echo "$user_key" >> /home/${username}/.ssh/authorized_keys
                # Add ssh key to root
                echo "$user_key" >> /root/.ssh/authorized_keys
                chmod 600 /home/${username}/.ssh/authorized_keys
                echo "$username: added ssh-key $key_comment"
            else
                echo "$username: ssh-key $key_comment already exists"
            fi
        fi
    done
else
    echo "No keys found"
fi

It seems as though it doesn't work or isn't enough. Do you have any suggestions?

Thanks!

[fkorotkov]

Now you can platform: netbsd to your compute_engine_instance to identify which version of the agent to use. I just tested it with your VM and it seems it doesn't respect startup-script/shutdown-script metadata that GCE VMs usually respect (like the Linux ones)?

[nbyavuz]

Hi @fkorotkov, could you retry? They should respect startup-script/shutdown-script metadata now.

[fkorotkov]

@nbyavuz tried it but don't see a difference. You can try it yourselves. Here is my config. But you can do the same with a gce_instance so you can check the VM in your project.

[nbyavuz]

Hi, openBSD / netBSD are working now, thanks! But although CI runs without any error, it exits with: Fatal exception on the agent! runtime error: invalid memory address or nil pointer dereference error. Does that error come from our side or CI side?

Example run link

[edigaryev]

Hi, openBSD / netBSD are working now, thanks! But although CI runs without any error, it exits with: Fatal exception on the agent! runtime error: invalid memory address or nil pointer dereference error. Does that error come from our side or CI side?

Example run link

This was fixed in cirruslabs/cirrus-ci-agent#219 and the new agent with this change was recently deployed.

[anarazel]

This was fixed in cirruslabs/cirrus-ci-agent#219 and the new agent with this change was recently deployed.

Awesome. Looks like this ticket could be closed then.

If there's interest, we might be able to create a fairly "bare" base image for net/openbsd, rather than one with all of postgres' dependencies installed. For others to use as a base image.