CI for Ruby on Rails: GitHub Actions vs. CircleCI

This is part three of a three part series (part 1, part 2) where I will walk you through setting up your CI suite with GitHub Actions, CircleCI, and then comparing which you may want to use if you are setting up continuous integration for your Rails app.

Part 3: Comparison

If you’re a developer like me, you probably experience a degree of FOMO (fear of missing out) whenever cool new tools are released. This was exactly how I felt when GitHub Actions were initially released. Since I was not in the beta until close to the offical release, I was worried I was missing out on a crazy innovative tool that would revolutionize my workflow.

I was eventually let into the beta, and have continued to play with GitHub Action’s since their public release, and I believe this is FOMO you can ignore. GitHub Action’s are a cool new tool in the toolbox, but don’t be so quick to give up on services like CircleCI.

Here’s why:

Resources

CircleCI has been around for awhile, and there are lots of amazing articles for setting it up with Ruby on Rails (like this thoughtbot article). Because of the available resources, it is pretty easy to get it up and running. GitHub Actions is the opposite. Because it is new, there isn’t a lot of content around setting it up, and you may run into an issue that will cause you to waste your time. Admittedly, it took me several evenings to get GitHub Actions running on CodeFund’s codebase. CircleCI also has tools like their local CLI which lets you verify your config file, and even run your workflow locally. Since no such tool exists for GitHub Actions, the only way to test your workflow is to keep pushing to GitHub until it works, which leads to an incredibly long, and aggravating, feedback cycle.

Speed

Here is the raw speed data comparing GitHub Actions & CircleCI for a CI suite that accomplishes the same tasks. Note that these times can vary slighly depending on a variety of factors, installing gems in particular can differ by over 30 seconds on no cache. These times were not cherry picked, you can view the raw logs here. ?

CircleCI – no cache: 9m 48s

Spin up Environment: 10s
Container circleci/postgres:11.2: 9m 38s
Checkout code: 7s
sudo apt-get update sudo apt-get install -y postgresql-client curl -o- -L https://yarnpkg.com/install.sh | bash: 5s
restore gem cache: 0s
restore yarn cache: 0s
set up assets cache key: 0s
restore assets cache: 0s
install gem dependencies: 4m 1s
install yarn dependencies: 30s
precompile assets: 1m 22s
save gem cache: 21s
save yarn cache: 13s
save assets cache: 1s
set up DB: 5s
run tests: 2m 29s
Uploading artifacts: 0s
run standardrb check: 5s
run ERB lint check: 11s
run prettier-standard check: 1s

CircleCI – with cache: 3m 47s

Spin up Environment: 11s
Container circleci/postgres:11.2: 3m 35s
Checkout code: 8s
sudo apt-get update sudo apt-get install -y postgresql-client curl -o- -L https://yarnpkg.com/install.sh | bash: 4s
restore gem cache: 6s
restore yarn cache: 4s
set up assets cache key: 0s
restore assets cache: 0s
install gem dependencies: 1s
install yarn dependencies: 12s
precompile assets: 13s
save gem cache: 0s
save yarn cache: 0s
save assets cache: 0s
set up DB: 6s
run tests: 2m 18s
Uploading artifacts: 0s
run standardrb check: 5s
run ERB lint check: 10s
run prettier-standard check: 1s

GitHub Actions – no cache: 10m 59s

Set up job: 2s
Initialize containers: 1m 3s
Run actions/[email protected]: 5s
Extract branch name: 0s
Get Yarn Cache: 1s
Cache Node Modules: 0s
Cache Gems: 1s
Cache Assets: 0s
Bundle Install: 3m 47s
Yarn Install: 41s
Compile Assets: 1m 58s
Setup DB: 7s
Run Rails Tests: 2m 15s
StandardRB Check: 9s
Prettier-Standard Check: 3s
Post Cache Assets: 2s
Post Cache Gems: 22s
Post Cache Node Modules: 20s
Post actions/[email protected]: 0s
Stop containers: 3s
Complete job: 0s
   

GitHub Actions – with cache: 7m 0s

Set up job: 3s
Initialize containers: 1m 8s
Run actions/[email protected]: 5s
Extract branch name: 0s
Get Yarn Cache: 1s
Cache Node Modules: 4s
Cache Gems: 7s
Cache Assets: 1s
Bundle Install: 3s
Yarn Install: 23s
Compile Assets: 2m 6s
Setup DB: 8s
Run Rails Tests: 2m 31s
StandardRB Check: 10s
Prettier-Standard: Check 3s
Post Cache Assets: 1s
Post Cache Gems: 1s
Post Cache Node Modules: 1s
Post actions/[email protected]: 1s
Stop containers: 3s
Complete job: 1s

To summarize, CircleCI is over a minute faster without cache and over three minutes with cache. If you are running multiple builds a day, that number quickly adds up.

Overall experience

I have to be honest, it took me a lot of time to get GitHub Action’s working with CodeFund’s codebase. During this time, I had to write my own Docker image, which I was lucky enough to already have some experience with. For beginners, or those unfamiliar with Docker, this is not a great experience, and even with experience, it still led to some very frustrating evenings.

If you followed along with the last two posts in this series, you will notice I did not have to change any code in my application to get CircleCI running, whereas I did with GitHub Actions.

CircleCI has great documentation, it has been rigorously battle tested, and it just works. There are lots of official and unofficial examples of how to get it to work best for your Rails app compared to GitHub Actions. CircleCI also has an awesome team that can help you through your issues.

Summary

GitHub Action’s may be the cool, new kid on the block, but I would not suggest replacing services like CircleCI with it. For me, I would consider using GitHub Actions for tasks like linting, autoformatting, or anything that needs to interact with the GitHub API. As far as your main continuous integration tool, I think it would be a mistake to replace CircleCI with GitHub Actions at this time. The speed difference is enough to dissuade me, but the bigger reason is how hard it is to work with GitHub Actions for a Rails app right now.

After this exploration, we have decided to stick with CircleCI at CodeFund.

Previous

Next

Share This