In this brief post, we present how we automate the testing of torokernel repository by using travis-ci. The tests require to set a host with KVM/QEMU and microvm enabled together with virtio-fs and virtio-vsock. This is not possible by using travis-ci VM, instead, it is necessary to use a remote host to launch the tests. In this post, we overview how tests are implemented in Toro and the challenges we face to deploy them in a remote host.
In Toro, tests are stored in the torokernel/tests directory and test the behavior of the units interface with respect to the specification. These tests are triggered when someone pushes to a branch and allow to quickly verify if the new commit is introducing a new bug or a performance regressions. Tests are split into folders depending on the unit that the test tests.
There are two types of tests: benchmarking and correctness. The benchmarking tests test the booting and kernel initialization time. The main goal is to avoid committing something that could impact in the time that toro takes to boot or initialize the kernel. The correctness tests test that the behavior of the different units interface corresponds with the specification. Also, they test corner cases of some APIs.
Tests are programs that are compiled within torokernel. They run several tests, and exit with either success or failure. This is gathered by the travis job that informs the user who launched the tests. Tests require to run as a KVM/QEMU and by using the microvm machine. This is not possible with current travis infrastructure. It is thus necessary to run the tests in a remote machine with KVM/QEMU installed. To do this, the travis-ci sshes to a remote machine in which it clones the corresponding repository and launches the tests. To setup this, it is necessary that the travis machine contains the private key that allows to ssh to the remote machine. This is done by encrypting the private key by using travis. Then, when the job is launched, travis descripts the private key and then it sshes to the remote machine that contains the public key.
We use as a remote host a s1-2 host in OVH. This host costs 3.5 dolar/month and allows us to install KVM. It has only 1 VCPU and 8 Gb of memory. Before the host can be used, it is necessary to install all the required dependencies, e.g., KVM, QEMU, socat, virtio-fs, etc. This is done by the script at https://github.com/torokernel/torokernel/blob/master/ci/prepare_host.sh. You can use it also to prepare a debian-10 host to compile and deploy toro microvm guest.
https://github.com/torokernel/torokernel/blob/master/ci/prepare_host.sh
https://gist.github.com/nickbclifford/16c5be884c8a15dca02dca09f65f97bd
https://oncletom.io/2016/travis-ssh-deploy/
