CircleCI Alternative for tests with Docker Compose


Summary:

  1. Why is CI/CD important?
  2. Is CircleCI the best CI/CD solution for your business?
  3. webapp.io capabilities missing in CircleCI
  4. CircleCI cons
  5. CircleCI vs. webapp.io CI/CD comparison summary
  6. Why businesses switch to webapp.io
  7. How to switch from CircleCI to webapp.io

1. Why is CI/CD important?

You'll notice a correlation between the fastest-growing companies in the world and great DevOps practices. Companies like Netflix, Amazon, Facebook, Etsy and Target are examples of this. CI/CD (what is CI?) is the first step to setting up a modern DevOps culture for high-performing software organizations.

2. Is CircleCI the best CI/CD solution for your business?

What is CircleCI?

CircleCI was founded in 2011 by Paul Biggar and Allen Rohner as a Continuous Integration and Continuous Delivery (CI/CD) platform after graduating from the 500 Startups program. CircleCI helps software teams rapidly release code by automating the build, test, and deploy process.

How much is CircleCI?

CircleCI pricing: as of March 2021, the highlights of CircleCI's Performance plan pricing is as follows (their billing FAQ and pricing page here):

  • Starts at $30/month
  • Up to 80 concurrent job runs
  • First 3 users is $15, then $15 per additional user
  • $15 per credit package of 25,000 credits
  • On the Performance plan, when you reach 2% of your remaining credits, you will be refilled 25% of your credit subscription, with a minimum refill of 25,000 credits
  • Purchased credits are applied to your machine usage
  • The default 2 vCPU and 4GB memory machine costs 10 credits per minute ($0.006). 25,000/10 = 2500 minutes. Assume 10 mins per build, 250 builds per month or 750 if concurrent jobs
  • Docker Layer Caching (DLC) uses additional 200 credits per job run

3. webapp.io capabilities missing in CircleCI

webapp.io logo

Why should you use webapp.io instead of CircleCI as your CI/CD vendor? webapp.io has many powerful features for modern CI/CD pipelines built for Docker Compose users:

webapp.io CapabilityFeature Description
Memory snapshotting to cache comprehensive test pipelineswebapp.io is the only CI vendor on the market with memory snapshotting capabilities that effectively cache more steps than CircleCI.
On-demand staging environments - preview environments before mergePreview changes faster with webapp.io full stack preview environments that are high-fidelity copies of your production environments you can use for stakeholder demos, code reviews.
Persistent staging environments on your domainwebapp.io Deployments is an extension of the Preview Environments feature.
SSH debugging terminal makes it easier to find and fix issues right in the test pipelinewebapp.io's SSH debugging terminal lasts for days after the test pipeline ran, making it easier to go back and fix issues right from within the testing dashboard without having to rerun the pipeline.
Version control system (VCS) GitLab supportwebapp.io supports GitLab users, a VCS CircleCI currently does not support.
Parallelism supportwebapp.io's infrastructure supports 12 parallel runs per developer, meaning a 15 person team gets 240 parallel runs at a time on webapp.io, while they get 80 parallel runs with CircleCI.

4. CircleCI cons

There are some key gaps with CircleCI that are revealed through feedback on online forums, review sites, and more. Below is a list that explains each gap, and the direct source. This information was taken on March 11, 2021, and we have made a vendor comparison to give anchoring context.

#1: CircleCI UI experience

The CircleCI 2.0 experience is quite complex and seems to be the biggest piece of criticism from their users. Check out this Sept 2020 review on Capterra

You can see a screenshot of their live React example below:


This is where webapp.io shines. It's much easier to search for your pipelines and look at parallel/concurrent runs in a more intuitive CI dashboard for your developer team.


#2: CircleCI YAML file complexity

Configuration files specific to CircleCI are pretty complex in nature. Here's a review from January 2021 that gives good context to what a maintainer/implementor of CircleCI should expect. CircleCI Orbs help make these YAML files a bit more concise for common steps, however you would still be required to learn a new syntax that promotes vendor lock-in.

Here's an example of an optimized Docker Compose CircleCI YAML file (205 lines)

Here's an example of an optimized Docker Compose webapp.io Layerfile (25 lines)


#3: CircleCI pricing model complexities

Since October 2019, CircleCI announced a change in pricing incentives has caused a stir for loyal users in the developer community such as this post here given the complexity in their cost structure.

At first glance, their pricing looks reasonable. When we compare this to other CI vendors it starts to look a little more expensive than it should be.

The pricing model for CircleCI's Performance Plan is best shown through an example side-by-side comparison to webapp.io's Team Plan pricing. Imagine you run Acme Corp, and you have the following requirements for your CI/CD vendor:

  • 15 person software developer team
  • Needs vCPU + Memory/Ram resources provided by CircleCI's Docker Linux VM - size XLarge
  • Needs Docker Layer Caching (DLC)
  • Assumes running 80 pipelines a day, 20 working days a month (80x20 = 1600 pipelines per month)
  • Assumes 15 minutes per pipeline

This analysis does not include the cost of hosted staging servers (included in webapp.io), ongoing maintenance fees, and other variable costs that are a bit more difficult to do an apples-to-apples comparison.

CircleCI Performance Plan vs. webapp.io Team Plan Cost Analysis

Monthly RequirementsCircleCIwebapp.ioCost of CircleCICost of webapp.io
Per active developer price$15 for the first 3, then $15/developer$35/developer$195/mo$525/mo
Docker Layer Caching (DLC)Extra spendIncluded200 credits per job run$0/mo
Docker Linux VMLarge = 4 vCPUs, 8 GB memory/ram6 vCPUs, 16 GB memory/ram20 credits/minute$0/mo
# of Concurrent Runs80 runs per team180 for 15 person team$0/mo$0/mo
Technical Customer SupportPaid 8x5 level support (Silver Package)Included 8x5 level support in every team package$249/mo$0/mo
Minimum credit purchaseAt least $15 for 25K credits, then 25% refill each timeNo credit system needed

320K credits for DLC per month = 12.8 x 25K credit packages

480K credits for pipelines per month = 19.2 x 25K credit packages



$0/mo
Total cost
per month


$924/mo$525/mo
Total cost per year

$11,340/yr$6300/yr


5. CircleCI vs. webapp.io CI/CD Comparison Summary

PerformanceCircleCIwebapp.io
CI/CD Platform
More than 80x concurrent runs
per team


Docker Layer Caching (DLC)
Memory Snapshotting
Details on duration for each command execution


Browsable artifacts and secret management


Admin Permissions Management


Observability

Full stack Preview Environments for visual reviews and staging
Persistent Staging Deployments on your own domain that autostop
E2E test recordings (with Cypress, Percy, Reflect.run) automatically viewable in pipelines


SSH debugging terminal available for days after pipeline runs


Optionality

Hosted SaaS Version

Pricing and usage predictability

GitHub Support
BitBucket Support
GitLab + Self-Hosted GitLab Support
Slack Integration

Admin Permissions Management
Linux VM Support

Docker VM Support

MacOS VM Support + Windows VM Support

Private Cloud Support
On-Prem Support


Multirepo and Polyrepo support


Free 8x5 customer support (silver support)
Out of the box support for most popular frameworks (Ruby on Rails, Django, React)


6. Why businesses switch from CircleCI to webapp.io

We love serving innovative organizations around the world that are switching to modern CI practices to accelerate their engineering team and optimize their development workflow to achieve more in less time. Don't just take our word for it, hear from CircleCI users who have switched to webapp.io:

7. How to switch from CircleCI to webapp.io

Looking for the best CircleCI alternative? Simply start a free trial of webapp.io here.

If you'd like CircleCI migration support or are looking for the enterprise plan, feel free to email support@webapp.io and we're happy to provide additional resources.