Cirrus CI is the best CI system out there
Friday, 16 August 2024
Disclaimer
I’m in no way affiliated with Cirrus CI, apart from submitting a few PRs to their repos. I’m just genuinely amazed at how good it is.
A rant-ish intro
(feel free to skip if you’re not into that)
I’m a fan of CI. I love seeing green check next to my PRs. The thing is, “modern” CI systems mostly suck.
Travis CI commoditized CI but has been enshittified to the ground1. The vast majority of open-source software has since moved to GitHub Actions, and it seems like it’s here to stay for longer. Unlimited minutes are too good to ignore, right?
It’s becoming indispensable. But think about this: how much longer will Microsoft want to pay for running CI for millions of projects out there? I’m pretty sure they’ll stop it at some point, and actually start milking money out of us. What else would they try to capture so much market share?
Apart from that, I think there are solid technical arguments against GitHub Actions. It has shaky fundamentals. Many things are harder than they should be; other are unsupported (e.g. local execution). I mean, just go watch this great video by fasterthanlime (it’s both scary and very funny).
In this blogpost, I’ll try to convince you that Cirrus CI is objectively better than every other major CI system in existence: GitHub Actions, CircleCI, GitLab CI/CD, and Buildkite.
Why is Cirrus CI the best
(These are the reasons that matter to me. I might’ve forgotten about some)
Local execution support
The nemesis of GitHub Actions, GitLab CI/CD, CircleCI, and honestly, pretty much everything else. And no, nektos/act isn’t good enough.
How did it happen that in 2024 we still cannot easily run CI pipelines locally? We send our jobs to the master in the cloud and pray for the green checkmark.
Cirrus CI provides the AGPL-licensed Cirrus CLI
tool, a static binary written in Go. Just
brew install cirruslabs/cli/cirrus
and you can run
your CI jobs locally:
cirrus run test
will run the task test
in a fresh Docker
container:
task:
name: test
container:
image: azul/zulu-openjdk-alpine:21
lint_script: ./gradlew detekt
test_script: ./gradlew test
I’m not going to explain the YAML config format of Cirrus CI, it’s not the focus of this post. It’s very similar to GitHub Actions or Circle CI.
Or maybe (assuming you’re on macOS) you want to run a macOS VM? No problem!
test_macos_task:
name: Run `maestro test` on macOS
macos_instance:
image: ghcr.io/cirruslabs/macos-sonoma-xcode:latest
This uses Tart - a source-available, very convenient wrapper around Apple’s Virtualization.framework. Guess what, it’s also created by Cirrus CI.
Persistent workers
What if you don’t want to run your CI jobs in the cloud, but on your own hardware? Maybe, I don’t know, your job is actually cool and you do some embedded development, you have a server that’s connected to some MCUs in your space lab, and you’d like to run tests on it whenever you push to a branch.
Cirrus CI lets you create a persistent worker to do just that! Really, it doesn’t get any easier. If your use-case is exotic (and you understand the tradeoffs), you can even choose to not spawn a new Docker container for every task run, but run directly the server, to not install dependencies for the 1000th time.
This single feature seems similar to the entire premise of Buildkite – run agents on your own infra. In Cirrus CI, it’s just another feature!
Open-source
Rest assured, I’m not here with another idealistic “big corp bad, do the right thing, use only open-source, hurr durr” rant.
The thing is, being open-source is an actual feature. I can look into the Cirrus CLI or Cirrus Agent code. I can fix things that annoy me.
It’s also very interesting from the educational standpoint. I like peeking under the hood of things.
In addition to Cirrus CLI being open-source, Cirrus CI actively innovates in the tooling space by creating source-available tools like Tart, Vetu, and Orchard. Those are impressive tools on their own, but integrate very well with Cirrus CI.
Config that scales
Sooner or later CI configuration becomes an incomprehensible mess of duplicated YAML. There are different approaches to solve that, but Cirrus CI does the best job here again.
In addition to YAML, we can also define our jobs in Starlark – a
tiny, deterministic language that’s similar (both in syntax and
semantics) to Python. Starlark code for Cirrus is written in the
.cirrus.star
file in the repo root. Here are docs
for programming Cirrus tasks in Starlark.
Let’s say you have 10 tasks that all run the same
apt-get install
as their first step? Easy – simply
extract those calls to a function and put it in some
common.star
file:
# common.star
def install_stuff():
return script(
"install_stuff",
"apt-get install bar",
'apt-get install whatever-you-want',
)
and then call that function 10 times, just like you’d do in normal code:
# .cirrus.star
"common.star", install_stuff) # import our common utils
load(
def main():
= fs.read("pubspec.yaml")
pubspec = yaml.loads(pubspec)["environment"]["flutter"]
flutter_version
return [
task(= "Run thingies",
name = container(image = "node:22-alpine.3.19"),
instance = [
instructions
install_stuff(),# ...
],
), ]
Or better even - instead of installing it 10 times, create a Dockerfile and install the dependencies there. Cirrus CI will automatically build an image, cache it, and use it for subsequent runs (see docs). How cool is that!
With Starlark, you can also generate the CI pipeline code
dynamically. Here’s an example that uses the Flutter version
directly from pubspec.yaml
(package.json
but in Flutter world), and uses that to pull the matching OCI
image:
# .cirrus.star
"cirrus", "fs", "yaml")
load(
def main():
= fs.read("pubspec.yaml")
pubspec = yaml.loads(pubspec)["environment"]["flutter"]
flutter_version
return [
task(= "Build Android app",
name = "build_andrid",
alias = container(
instance = "ghcr.io/cirruslabs/flutter:%s" % flutter_version,
image
),= [
instructions "pub", "~/.pub-cache"),
cache("flutter", "build", "apk"),
script(# ...
],
), ]
This is a very efficient and pleasant approach to generating workflows dynamically. No more YAML!
And if this wasn’t impressive enough, Cirrus CI config can be locally validated and tested – which brings us to the next point.
Config validation and testing
How many times did you git push
only to see
this
Sure, now there’s a GitHub Actions extension for VSCode that makes makes it easier to write correct workflows thanks to its integration with JSON schema. But it’s only in VSCode.
Want to validate that your Cirrus CI files (both in YAML and in Starlark) don’t contain syntactic (and some semantics) errors?
cirrus validate
Why no other CI does this? So simple, so useful.
Simple and modern
Cirrus CI is actually well thought of and simple (see Life of a Build). They don’t maintain their own server fleet – instead they run your tasks on public clouds like GCP, AWS, and Azure.
No magic. It’s all OCI all the way down!
You may think that GitHub Actions is also simple and obvious - it is not2. Don’t look under the hood if you want to feel good about using it.
Flexible
If you didn’t already realize that, Cirrus CI is very flexible. Linux container? Linux Arm container? MacOS VM? FreeBSD VM? Your own OCI-compatible Linux container or macOS VM image, running on your own infra? Check.
It even supports Windows Containers - a thing I didn’t know exists (and which I’ve never used).
I don’t know what else you might want.
What to be aware of
Nothing is perfect, and neither is Cirrus CI – but it definitely does come the closest to some “Continuous Integration singularity”.
The web console UI is very basic, but honestly, I don’t care at all – it does the job.
It’s not completely free for open-source, like GitHub Actions. The free plan is generous though - 10 000 CPU-minutes for Linux tasks or 500 minutes for macOS tasks (which always use 4 CPUs). See pricing.
It’s not priviliged like GitHub Actions, so you don’t have access to
$GITHUB_TOKEN
automatically. Gotta generate that PAT.For private personal repos, it costs $10/month (though I think it’s a very fair price). For private org repos, it’s $10/seat/month.
It integrates only with GitHub.
Low bus factor of the company (/s), which leads us to…
Who’s behind it?
From what I see on Cirrus’ GitHub repos, it’s built by literally 2 guys – Fedor Korotkov, a former Airbnb and JetBrains employee, and Nikolay Edigaryev.
Those two are single-handedly revolutionizing the CI space.
The sad thing to me is that Cirrus CI isn’t more popular. So many people accept the (arguably pretty shitty) status quo of CI. It can be so much better, and Cirrus CI shows it’s possible.
Who uses it?
If you’re a solo developer, I hope you’re sold by now. But if you’re a company (or a solo developer, just a bit less adventurous), you might be thinking: okay, this Cirrus CI thing is all fine and dandy, but does anyone actually use it?
Cirrus CI is used by projects such as FreeBSD, PostgreSQL, Podman, and Bitcoin. It was also used as part of Flutter’s CI infra for many years, but Google being Google, they decided to use only their infra and ditched it.
Further reading
- I’m mentioning it the 3rd time now, but it’s really worth it: go watch GitHub Actions feels bad, you will not regret!
- Introducing Cirrus CI
- Cirrus CI stack
- Core principle of Continuous Integration systems is obsolete
Summing up
It truly is amazing what a small team can accomplish by focusing on a problem and just solving it, the right way. Cirrus CI completely out-executed major players like Microsoft, GitLab, and CircleCI.
I encourage everyone angry at their CI to give Cirrus CI a try. It’s truly a breath of fresh air.
2024-08-18 – frp (Fast Reverse Proxy)
instead of ngrok
2024-08-16 – Cirrus CI is the best CI system
out there
2024-08-14 – Going to Berlin for
Droidcon/Fluttercon
2024-06-25 – I was awarded Google Open Source Peer
Bonus
2024-06-04 – My journey to Google I/O
’24
2024-05-11 – GitHub Actions
beg for a supply chain attack
2024-03-19 – Writing a custom Dart
VM service extension (part 1)
2024-02-08 – On using smartphone for things
that make sense
2023-11-30 – Semantics in Flutter - under
the hood
2023-11-25 – Flutter Engine notes
2023-09-17 – Creating
and managing Android Virtual Devices using the terminal
2023-05-27 – Suckless
Android SDK setup
2023-05-26 – Let’s start over
2023-05-21 –
Short thought on “The Zen of
Unix”
2023-05-15 – Notes about “flutter
assemble”
2019-01-07 – Google Code-in
2018