CircleCI Alternative for tests with Docker Compose
Summary:
- Why is CI/CD important?
- Is CircleCI the best CI/CD solution for your business?
- webapp.io capabilities missing in CircleCI
- CircleCI cons
- CircleCI vs. webapp.io CI/CD comparison summary
- Why businesses switch to webapp.io
- How to switch from CircleCI to webapp.io
1. Why is CI/CD important?
You'll notice a correlation between the fastest-growing companies in the world and great DevOps practices. Companies like Netflix, Amazon, Facebook, Etsy and Target are examples of this. CI/CD (what is CI?) is the first step to setting up a modern DevOps culture for high-performing software organizations.
2. Is CircleCI the best CI/CD solution for your business?
What is CircleCI?
CircleCI was founded in 2011 by Paul Biggar and Allen Rohner as a Continuous Integration and Continuous Delivery (CI/CD) platform after graduating from the 500 Startups program. CircleCI helps software teams rapidly release code by automating the build, test, and deploy process.
How much is CircleCI?
CircleCI pricing: as of March 2021, the highlights of CircleCI's Performance plan pricing is as follows (their billing FAQ and pricing page here):
- Starts at $30/month
- Up to 80 concurrent job runs
- First 3 users is $15, then $15 per additional user
- $15 per credit package of 25,000 credits
- On the Performance plan, when you reach 2% of your remaining credits, you will be refilled 25% of your credit subscription, with a minimum refill of 25,000 credits
- Purchased credits are applied to your machine usage
- The default 2 vCPU and 4GB memory machine costs 10 credits per minute ($0.006). 25,000/10 = 2500 minutes. Assume 10 mins per build, 250 builds per month or 750 if concurrent jobs
- Docker Layer Caching (DLC) uses additional 200 credits per job run
3. webapp.io capabilities missing in CircleCI
Why should you use webapp.io instead of CircleCI as your CI/CD vendor? webapp.io has many powerful features for modern CI/CD pipelines built for Docker Compose users:
| webapp.io Capability | Feature Description |
|---|---|
| Memory snapshotting to cache comprehensive test pipelines | webapp.io is the only CI vendor on the market with memory snapshotting capabilities that effectively cache more steps than CircleCI. |
| On-demand staging environments - preview environments before merge | Preview changes faster with webapp.io full stack preview environments that are high-fidelity copies of your production environments you can use for stakeholder demos, code reviews. |
| Persistent staging environments on your domain | webapp.io Deployments is an extension of the Preview Environments feature. |
| SSH debugging terminal makes it easier to find and fix issues right in the test pipeline | webapp.io's SSH debugging terminal lasts for days after the test pipeline ran, making it easier to go back and fix issues right from within the testing dashboard without having to rerun the pipeline. |
| Version control system (VCS) GitLab support | webapp.io supports GitLab users, a VCS CircleCI currently does not support. |
| Parallelism support | webapp.io's infrastructure supports 12 parallel runs per developer, meaning a 15 person team gets 240 parallel runs at a time on webapp.io, while they get 80 parallel runs with CircleCI. |
4. CircleCI cons
There are some key gaps with CircleCI that are revealed through feedback on online forums, review sites, and more. Below is a list that explains each gap, and the direct source. This information was taken on March 11, 2021, and we have made a vendor comparison to give anchoring context.
#1: CircleCI UI experience
The CircleCI 2.0 experience is quite complex and seems to be the biggest piece of criticism from their users. Check out this Sept 2020 review on Capterra
You can see a screenshot of their live React example below:
This is where webapp.io shines. It's much easier to search for your pipelines and look at parallel/concurrent runs in a more intuitive CI dashboard for your developer team.
#2: CircleCI YAML file complexity
Configuration files specific to CircleCI are pretty complex in nature. Here's a review from January 2021 that gives good context to what a maintainer/implementor of CircleCI should expect. CircleCI Orbs help make these YAML files a bit more concise for common steps, however you would still be required to learn a new syntax that promotes vendor lock-in.
Here's an example of an optimized Docker Compose CircleCI YAML file (205 lines)
Here's an example of an optimized Docker Compose webapp.io Layerfile (25 lines)
#3: CircleCI pricing model complexities
Since October 2019, CircleCI announced a change in pricing incentives has caused a stir for loyal users in the developer community such as this post here given the complexity in their cost structure.
At first glance, their pricing looks reasonable. When we compare this to other CI vendors it starts to look a little more expensive than it should be.
The pricing model for CircleCI's Performance Plan is best shown through an example side-by-side comparison to webapp.io's Team Plan pricing. Imagine you run Acme Corp, and you have the following requirements for your CI/CD vendor:
- 15 person software developer team
- Needs vCPU + Memory/Ram resources provided by CircleCI's Docker Linux VM - size XLarge
- Needs Docker Layer Caching (DLC)
- Assumes running 80 pipelines a day, 20 working days a month (80x20 = 1600 pipelines per month)
- Assumes 15 minutes per pipeline
This analysis does not include the cost of hosted staging servers (included in webapp.io), ongoing maintenance fees, and other variable costs that are a bit more difficult to do an apples-to-apples comparison.
CircleCI Performance Plan vs. webapp.io Team Plan Cost Analysis
| Monthly Requirements | CircleCI | webapp.io | Cost of CircleCI | Cost of webapp.io |
| Per active developer price | $15 for the first 3, then $15/developer | $35/developer | $195/mo | $525/mo |
| Docker Layer Caching (DLC) | Extra spend | Included | 200 credits per job run | $0/mo |
| Docker Linux VM | Large = 4 vCPUs, 8 GB memory/ram | 6 vCPUs, 16 GB memory/ram | 20 credits/minute | $0/mo |
| # of Concurrent Runs | 80 runs per team | 180 for 15 person team | $0/mo | $0/mo |
| Technical Customer Support | Paid 8x5 level support (Silver Package) | Included 8x5 level support in every team package | $249/mo | $0/mo |
| Minimum credit purchase | At least $15 for 25K credits, then 25% refill each time | No credit system needed | 320K credits for DLC per month = 12.8 x 25K credit packages 480K credits for pipelines per month = 19.2 x 25K credit packages | $0/mo |
| Total cost per month | $924/mo | $525/mo | ||
| Total cost per year | $11,340/yr | $6300/yr |
5. CircleCI vs. webapp.io CI/CD Comparison Summary
| Performance | CircleCI | webapp.io |
| CI/CD Platform | ✅ | ✅ |
| More than 80x concurrent runs per team | ❌ | ✅ |
| Docker Layer Caching (DLC) | ✅ | ✅ |
| Memory Snapshotting | ❌ | ✅ |
| Details on duration for each command execution | ✅ | ✅ |
| Browsable artifacts and secret management | ✅ | ✅ |
| Admin Permissions Management | ✅ | ✅ |
| Observability | ||
| Full stack Preview Environments for visual reviews and staging | ❌ | ✅ |
| Persistent Staging Deployments on your own domain that autostop | ❌ | ✅ |
| E2E test recordings (with Cypress, Percy, Reflect.run) automatically viewable in pipelines | ❌ | ✅ |
| SSH debugging terminal available for days after pipeline runs | ❌ | ✅ |
| Optionality | ||
| Hosted SaaS Version | ✅ | ✅ |
| Pricing and usage predictability | ❌ | ✅ |
| GitHub Support | ✅ | ✅ |
| BitBucket Support | ✅ | ❌ |
| GitLab + Self-Hosted GitLab Support | ❌ | ✅ |
| Slack Integration | ✅ | ✅ |
| Admin Permissions Management | ✅ | ✅ |
| Linux VM Support | ✅ | ✅ |
| Docker VM Support | ✅ | ✅ |
| MacOS VM Support + Windows VM Support | ✅ | ❌ |
| Private Cloud Support | ✅ | ✅ |
| On-Prem Support | ✅ | ❌ |
Multirepo and Polyrepo support | ✅ | ✅ |
| Free 8x5 customer support (silver support) | ❌ | ✅ |
| Out of the box support for most popular frameworks (Ruby on Rails, Django, React) | ✅ | ✅ |
6. Why businesses switch from CircleCI to webapp.io
We love serving innovative organizations around the world that are switching to modern CI practices to accelerate their engineering team and optimize their development workflow to achieve more in less time. Don't just take our word for it, hear from CircleCI users who have switched to webapp.io:
7. How to switch from CircleCI to webapp.io
Looking for the best CircleCI alternative? Simply start a free trial of webapp.io here.
If you'd like CircleCI migration support or are looking for the enterprise plan, feel free to email support@webapp.io and we're happy to provide additional resources.