Add test to assert no host changes

[jodh-intel]

Overview

We need a test to assert that the container workload cannot modify the host in any way. As a minimum, the container workload must not be able to modify:

1. The host mount table.
2. Any files on the host not provided to the container.

(1) is easy enough, but (2) is less so.

ZFS to the rescue

However, Ubuntu 20.04 was released on 23rd April and supports a ZFS root install as an "experimental" feature. This makes the problem of (2) much simpler since we could write a test that runs on an Ubuntu 20.04 ZFS VM.

Idea for the new test

The test would need to guarantee nothing else was using the VM and would then:

1. Save the output of "mount" and "zfs mount".

2. Create a ZFS snapshot.

3. Create a container.

4. Make the container workload attempt to do nasty things to files and mounts in the container.

   Essentially, we need a container "fuzzer" that will run a variety of commands, trying to create, delete, modify, append, overwrite files within the confines of the container. All the attempts should be logged (with full paths).

5. Stop the container.

6. Create a 2nd ZFS snapshot.

7. Run "mount" and "zfs mount" and compare to the output captured in (1).

   There should be no differences.

8. Run "zfs diff" on the two snapshots.

   Again, there should be no differences.

Container fuzzer

We need to see if such a tool already exists. If not, we could either write something in shell script or maybe golang.

wdyt @bergwolf, @gnawux, @amshinde, @chavafg, @jcvenegas, @devimc?

References:

• vc: make host shared path readonly runtime#2713
• VMT: add KCSAs for CVE-2020-2024 and CVE-2020-2025 community#156

[devimc]

thanks @jodh-intel for raising this, I have some questions:

does the container run on top of the ZFS snapshot?

what files will you attemp to create, delete, modify, append or overwrite? tmpfs (/proc, /sys) ?

I like the idea, there are already some tests to check the host's integrity, can we improve them? are new tests really needed?

tests/integration/docker/cp_test.go

Line 74 in 1234c17

Context("check host path integrity", func() {

tests/integration/docker/run_test.go

Line 225 in 1234c17

var _ = Describe("Check read-only cgroup filesystem", func() {

[jodh-intel]

@devimc - the container will simply run on the host system and the two snapshots will detect any changes to any files on the host system. The container fuzzer can and probably should attempt to do "anything it can" - try everything. This is just an idea. If we already have tests, great. But it sounds like we need to update them to detect the scenarios in kata-containers/community#156?

[devimc]

@jodh-intel pardon my ignorance, what do you need to reproduce those cves? a malicious container? what's a malicious container? a container able to escape from the guest's namespaces?
do we have such container? - I guess that the "fuzzer" test will run at the same level than the agent?

[jodh-intel]

@devimc - sorry, this test isn't supposed to reproduce CVE's, but just try to do anything it can to changes files in the container and see if anything changes on the host. Clearly, we don't expect any host files to change, so let's assert that behaviour 😄

I guess that the "fuzzer" test will run at the same level than the agent?

That's a good question. Ideally, I guess we'd run it in the default set of namespaces, but we could also run it in the agent ones if, for example, this lands: kata-containers/agent#788.

[wainersm]

Hi @jodh-intel ,

A colleague quality engineer has found a bug on QEMU + kata-containers caused by a syscall crashing the virtiofs (hopefully fixed on QEMU 5.1). She used a syscalls fuzzer [1] from inside a container spawned with kata runtime. I thought this idea of "fuzzing things" from inside the container very interesting, so maybe that syscall fuzzer can be put together others to stress a container and then hopefully uncovering bugs.

[1] https://github.com/kernelslacker/trinity

[jodh-intel]

@wainersm - nice!

@GabyCT - it would be great if you could look at creating a fuzz test to run trinity inside a container (or maybe even use https://github.com/google/syzkaller as it runs the fuzzer inside a VM anyway). I don't see a timeout option for trinity so looks like you'd need to kill the workload after 'n' seconds/minutes.

[GabyCT]

Closing this issue as it is related with kata 1.x