Multi-node parallelization is a feature that allows runs to be started faster, pending to be mitigated, and larger environments to be supported. It splits the work of a job across multiple nodes, if necessary.
Welcome to our 3rd Launch Week, where each weekday we unveil a product or feature release that our team has been working on in the past three months. If you haven't already, head over here to subscribe for future product updates and launch weeks.
Product Release #3: Multi-Node Parallelization
Access level: general availability
Multi-node parallelization is a feature that allows runs to be started faster, pending to be mitigated, and larger environments to be supported. It splits the work of a job across multiple nodes, if necessary. It works by checking what nodes have the snapshot and then checking if the node that the snapshot is on has the resources to run the job. If the node has the resources, then it is run. If not, the snapshot is moved to a new node and runs there.
Below is a demo video of how it works from our engineer, Connor, who led the project.
Then vs. Now: What's Changed?
Previously, organizations were each held on one dedicated node. During peak hours, this method of scheduling jobs led to some pending, particularly for users with larger environments.
Now, jobs are attempted to be scheduled on the node that has the snapshot, but if that node is busy for 1 minute it will transport the snapshot to the least busy node. This means that users will only be limited by the whole capacity of webapp.io instead of the capacity of a single server. This has resulted in significantly reduced pending runs.
Top Benefits of Multi-Node Parallelization
- Speeds up starting time of runs
- Mitigates pending runs
- Supports larger environments
Why did you Build This Feature?
We built this feature after seeing the difference that directives like SPLIT have made to our users. In particular, we noticed a significant speed difference in users running Cypress tests in parallel using Docker Compose. We continue to try to unlock the full potential of the webapp.io servers and optimize performance for all of our users, continually finding ways to speed up runs for our customers.
Getting Started with Multi-node Parallelization
To enable multi-node parallelization, navigate to your organization tab and check the box titled “Use multi-node scheduling”, as seen below.
Ready to get started? Sign up for free here, get some runs going, and see the difference that multi-node scheduling makes!