Firecracker is the technology which powers Serverless, but it can be also be repurposed to run end-to-end tests quickly and cheaply.
End-to-end tests and CI
When you make changes to code in a repository, you usually want to make sure that you haven't broken core user flows like the sign up button.
To automatically check nothing is broken, people usually start a copy of their app within their CI pipeline and then run test browsers to click around. That way you can make sure your app still builds, starts, and mostly works after every new change.
For a typical full-stack webapp, these are the sorts of steps that run after every new change:
This pipeline works fine, but it's significantly more expensive and slow than it could be:
- Images have to be built from scratch, or constantly pushed/pulled from a registry somewhere
- The database has to be re-migrated and re-seeded every time
- It's hard to parallelize things, because the setup needs to re-run for every independent copy of the environment.
Faster and cheaper e2e tests using base images
Since the build steps, database, and deployment definitions rarely change between code changes, it'd be nice if we could have a "base image" of a machine which already had everything started.
This base image would have:
- A migrated and seeded database already running
- Build caches populated with files from an earlier build
- Images and large files pre-downloaded
- The repository itself pre-loaded
If we had such a base image, we could pull in our new change and build/start things in seconds.
For a long time this was technically infeasible, but there's an exciting new technology that facilitates this exact use-case.
Firecracker
Firecracker is the framework that powers AWS Lambda. It allows you to make "snapshots" of a running VM, which contain the entire disk/memory of the vm. When you restore one, you also restore all of the VM's files and processes.
We could create a script that created a firecracker VM & set it up to be an ideal base image. Then we could copy the file that contained the VM's files and processes,
Getting started with firecracker
It's actually relatively simple to use firecracker because of their quickstart guide and other amazing resources.
Using firecracker on Linux can be as simple as a few commands:
# Step 1. download firecracker
curl -fSsL https://github.com/firecracker-microvm/firecracker/releases/download/v1.0.0/firecracker-v1.0.0-x86_64.tgz | tar -xz
mv firecracker-* firecracker
# Step 2. Download files required to start the VM
image_bucket_url="https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/$arch"
wget "${image_bucket_url}/kernels/vmlinux.bin"
wget "${image_bucket_url}/rootfs/bionic.rootfs.ext4"
# Step 3. Create a config file
cat <<EOF > vmconfig.json
{
"boot-source": {
"kernel_image_path": "vmlinux.bin",
"boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
},
"drives": [
{
"drive_id": "rootfs",
"path_on_host": "bionic.rootfs.ext4",
"is_root_device": true,
"is_read_only": false
}
]
}
EOF
# Step 4. Start the VM
./firecracker --unix-socket /tmp/firecracker.socket --config-file vmconfig.json
Great, we have a VM, now what?
We wanted to use firecracker to create a "zygote snapshot" which contained all of our dependencies. If you're following along, use Julia Evans' tutorial to set up networking, and you should be able to ssh directly into your VM!
From there, run your setup commands as necessary:
Finally, exit the VM and take a snapshot! (see Firecracker docs)
curl --unix-socket /tmp/firecracker.socket -i \
-X PATCH 'http://localhost/vm'
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{"state": "Paused"}'
curl --unix-socket /tmp/firecracker.socket -i \
-X PUT 'http://localhost/snapshot/create' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"snapshot_type": "Full",
"snapshot_path": "./snapshot_file",
"mem_file_path": "./mem_file",
"version": "0.23.0"
}'
How it all fits together
Now that we have our disk and memory snapshot files, we can just run a simple script to prepare the VM to run our CI pipeline:
- Copy the zygote files: bionic.rootfs.ext4, snapshot_file, and mem_file to another directory
- Restore the snapshot files using the firecracker API
- Run the build for your pipeline within the restored VM
Benchmark
Enough about theory, let's look at a benchmark!
This benchmark uses an open source Slack clone, which uses Docker Compose, Go, PostgreSQL, and React.
Firecracker-based CI is 10.5x faster
This benchmark shows a drastic 10x speedup due to having a warm cache. The firecracker VM came up almost instantly, and ran the build steps in just a few seconds due to the reused caches. This meant that the whole pipeline could run in only 30s.
In comparison, the "traditional CI" VMs took over a minute just to boot and clone the code, and then had to re-build the images from scratch, before ever being able to run the tests.
See these links for methodology:
- GitHub actions build
- Firecracker build (notice the "skipped" steps)
- Commit which triggered the builds
Want to try firecracker-based CI yourself?
Webapp.io offers firecracker-based VMs for CI even on our free plan and is listed on the Firecracker homepage as an integrated technology. If you'd like to try out the approach mentioned in this post without setting it up yourself, you can create a webapp.io account and try a sample in just a few minutes at https://webapp.io.
Discussions: