Skip to content

Commit

Permalink
[CI] Fix virt-customize issue in vmtests
Browse files Browse the repository at this point in the history
In main branch vmtests seems broken with the following error:
Using: kernel:tests/vmtests/test-data/kernels/6.6/boot/vmlinuz-6.6.65
time="2024-12-16T16:25:58Z" level=info msg="starting to build images" queue=tetragon.qcow2
time="2024-12-16T16:25:58Z" level=info msg="starting command" args="[qemu-img convert -f qcow2 -O qcow2 tests/vmtests/test-data/images/base.qcow2 tests/vmtests/test-data/images/tetragon.qcow2]" cwd=/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon path=/usr/bin/qemu-img
time="2024-12-16T16:25:58Z" level=info msg="set current working dir to 'tests/vmtests/test-data/images/'"
time="2024-12-16T16:25:58Z" level=info msg="starting command" args="[virt-customize -a tetragon.qcow2 --hostname tetragon --append-line /etc/sysctl.d/local.conf:kernel.panic_on_rcu_stall=1 --mkdir /home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon --copy-in /tmp/tetragon-vmtests-1313171169/fstab:/etc --copy-in /home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/tests/vmtests/tetragon-tester:/sbin/ --copy-in /tmp/tetragon-vmtests-1313171169/tetragon-tester.json:/etc --copy-in /tmp/tetragon-vmtests-1313171169/tetragon-tester.service:/etc/systemd/system --run-command systemctl enable tetragon-tester.service --append-line /etc/ssh/sshd_config:PermitRootLogin yes --append-line /etc/ssh/sshd_config:PermitEmptyPasswords yes --copy-in /tmp/tetragon-vmtests-1313171169/20-interfaces.network:/etc/systemd/network --chmod 0644:/etc/systemd/network/20-interfaces.network --run-command systemctl enable systemd-networkd.service]" cwd=/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/tests/vmtests/test-data/images path=/usr/bin/virt-customize
time="2024-12-16T16:25:58Z" level=info msg="stdout> [   0.0] Examining the guest ...\n"
time="2024-12-16T16:26:09Z" level=warning msg="stderr> virt-customize: error: libguestfs error: passt exited with status 1\n"
time="2024-12-16T16:26:09Z" level=warning msg="stderr> \n"
time="2024-12-16T16:26:09Z" level=warning msg="stderr> If reporting bugs, run virt-customize with debugging enabled and include the complete output:\n"
time="2024-12-16T16:26:09Z" level=warning msg="stderr> \n"
time="2024-12-16T16:26:09Z" level=warning msg="stderr>   virt-customize -v -x [...]\n"
time="2024-12-16T16:26:09Z" level=error msg="error executing command" error="exit status 1" image=tetragon.qcow2
time="2024-12-16T16:26:09Z" level=warning msg="image file 'tests/vmtests/test-data/images/tetragon.qcow2' not deleted so that it can be inspected"
time="2024-12-16T16:26:09Z" level=warning msg="image build failed" image=tetragon.qcow2 queue= result="{Error:exit status 1 CachedImageUsed:false CachedImageDeleted:}"
Error: images errors:tetragon.qcow2: exit status 1

This happens from a recent update of libguestfs-tools from 1.46.2-10ubuntu3
to 1.52.0-5ubuntu3.

After enabling debug arguments in virt-customize we can see (full logs https://gist.github.com/tpapagian/e71fdd5f09e4eaa03d79e521a63bf435):
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: passt\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --one-off\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --socket /tmp/libguestfsGDkZz0/passt.sock\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --pid /tmp/libguestfsGDkZz0/passt3.pid\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --address 169.254.2.15\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --netmask 16\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --mac-addr 52:56:00:00:00:02\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: command: run: \\ --gateway 169.254.2.2\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> Don't run as root. Changing to nobody...\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> No routable interface for IPv6: IPv6 is disabled\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> Template interface: eth0 (IPv4)\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> MAC:\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     host: 52:56:00:00:00:02\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> DHCP:\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     assign: 169.254.2.15\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     mask: 255.255.0.0\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     router: 169.254.2.2\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> DNS:\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     169.254.2.2\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> DNS search list:\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     vfauej4skqkeff2vnl2tg2wrza.cx.internal.cloudapp.net\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> UNIX domain socket bound at /tmp/libguestfsGDkZz0/passt.sock\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> \n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> You can now start qemu (>= 7.2):\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     kvm ... -device virtio-net-pci,netdev=s -netdev stream,id=s,server=off,addr.type=unix,addr.path=/tmp/libguestfsGDkZz0/passt.sock\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> or qrap, for earlier qemu versions:\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr>     ./qrap 5 kvm ... -net socket,fd=5 -net nic,model=virtio\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> PID file open: Permission denied\n"
time="2024-12-17T07:44:12Z" level=warning msg="stderr> libguestfs: trace: launch = -1 (error)\n"

This is related to passt (https://passt.top/passt/) which for networking.
As we don't need networking in that step (i.e. no --install args in virt-customize)
we can disable that for now to unblock things.

We can revisit that in the future if needed.

Signed-off-by: Anastasios Papagiannis <anastasios.papagiannis@isovalent.com>
  • Loading branch information
tpapagian committed Dec 17, 2024
1 parent e4f084f commit 42f725c
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions cmd/tetragon-vmtests-run/image.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"path/filepath"

"github.com/cilium/little-vm-helper/pkg/images"
"github.com/cilium/little-vm-helper/pkg/step"
"github.com/cilium/tetragon/pkg/vmtests"
"github.com/sirupsen/logrus"
)
Expand Down Expand Up @@ -198,6 +199,19 @@ func buildNetActions(tmpDir string) ([]images.Action, error) {
return ret, nil
}

type NoNetworkCommand struct{}

func (rc *NoNetworkCommand) ActionOpName() string {
return "no-network"
}

func (rc *NoNetworkCommand) ToSteps(s *images.StepConf) ([]step.Step, error) {
return []step.Step{&images.VirtCustomizeStep{
StepConf: s,
Args: []string{"--no-network"},
}}, nil
}

func buildTestImage(log *logrus.Logger, rcnf *RunConf) error {

imagesDir, baseImage := filepath.Split(rcnf.baseImageFilename)
Expand All @@ -224,6 +238,7 @@ func buildTestImage(log *logrus.Logger, rcnf *RunConf) error {
}

actions := []images.Action{
{Op: &NoNetworkCommand{}},
{Op: &images.SetHostnameCommand{Hostname: rcnf.vmName}},
{Op: &images.AppendLineCommand{
File: "/etc/sysctl.d/local.conf",
Expand Down

0 comments on commit 42f725c

Please # to comment.