I’ve started a project! 🙌 In short, I’m building a Continuous Deployment platform. The inspiration for this project came from my experience with CD platforms over my professional career. I’ve never particularly been a detractor from the existing CD platforms out there, they all have their place and their own quirks.

CI/CD - What’s the difference?

This is almost embarrasing to share. I’ve done DevOps in some form now for 9 years. This is one of the core fundamentals of being a DevOps practitioner.

So first up, CI stands for Continuous Integration. Then CD stands for Continuous Deployment. You’ll see these terms used interchangeably, I am also guilty of this. It probably comes from the fact they appear as a single word in the acronym CI/CD. Suggesting they are interchangeable.

For the most part, you can do CD wherever you do CI. This is where the confusion comes in. CI platforms tend to be pretty free-form. Often allowing you to run any command you want. CD platforms are more prescriptive. They often have a specific way of doing things. But given CI so so free-form, you can usually deploy whatever you need and fulfill the CI/CD lifecycle.

What’s the point of a CD platform?

So why would you ever need a CD platform? Surely something more free-form would be better?

I didn’t - for a very long time. In fact I would encourage you to do the simple thing. GitHub actions, Gitlab CI, Jenkins, etc. They’re all fucking great tools and you can almost certainly do the job and do it well. Do that, especially if you’re just starting out.

There’s many reasons why you would, but I’ll stick to the main three:

1. You want deep integration with the target deployment platform
2. You don’t want to write the integration with the target platform
3. You want complex deployment strategies

Deep Target Platform Integration

Deep integration with a target platform is probably the most striking difference between hand-rolling your own CD inside a free-form CI platform. An example would be deploying an application to Kubernetes.

There’s several ways to deploy to kubernetes, you have YAML files, Helm charts, Kustomize, etc. What do they all have in common? They all come with a CLI. Realistically, that’s what a lot of platforms come with, that or raw API’s. This makes free-form a lot more complex.

You’ll want your pipeline to be consistent. Consistency is key for repeatability and trust in the pipeline. This requires systems like state machines to abstract over the underlying platforms. Even with Kubernetes, which boasts eventual consistency, you still want to distill down to a successful deployment or not - Kubernetes sucks at that, YMMV with other platforms. Try writing that state machine in bash? Decline.

You don’t want to write the integration with the target platform

This one is huge. OpenSource is built off the shoulders of kind individuals who share their time and expertise. There’s just not a huge demand for these types of CD platforms outside of the enterprise.

There are in fact some great OpenSource CD platforms out there. These often vary in maturity and scope. You have huge platforms like Spinakker which have existed for many years. There’s also more niche projects like ArgoCD which is quite domain specific. ArgoCD was actually one of the first CD platforms I used and started enlightening me to what a well crafted CD platform can and should look like.

There’s also CD abstracts like Tekton, which are less opinionated on the how and feel closer to a CI platform. I’ll be basing my project on Tekton.

Complex Deployment Strategies

Complex deployment strategies are more about the strategy than the actual deployment. Often the underlying deployment process is simple, but the strategy is complex due to size and or limitations of the underlying CI platform.

An exmaple of this would be a semi complex deployment in GitHub Actions. To start with, you can write a simple pipeline which deploys to a single environment. Great! Now lets deploy to multiple environments. You’d use something like a matrix:

jobs:
  deploy:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        env: [dev, demo, prod]

    steps:
      - name: Deploy to ${{ matrix.env }}
        run: |
          echo "Hello ${{ matrix.env }}"

That’s still fairly simple. Now let’s say you want to deploy to multiple regions. You’d use a matrix again:

jobs:
  deploy:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        env: [dev, demo, prod]
        regions: [us-east-1, eu-west-1]

    steps:
      - name: Deploy to ${{ matrix.env }} in ${{ matrix.region }}
        run: |
          echo "Hello ${{ matrix.env }} in ${{ matrix.region }}"

Okay, maybe I’m making this look too easy. But if we look at the detail, you’ll see that we’re actually deploying to dev and demo in both regions - which isn’t great. Okay, so we can still deal with that, GitHub Actions has includes & excludes, we can use a mixture of those. Not ideal but it’s a start.

Now let’s look at it again, we’re actually deploying to every stage in parallel. That’s okay, we can use reuseable workflows, we can even combine that with matrices for the regions.

Eh. Not my tempo 🥁

This all feels like an after thought and feels like a hack. Just because you can do it, it doesn’t mean you should. More so - even if you can do it and understand it, will others?

What’s Next?

In short, I’m building a CD platform. I’ll be documentating the process of building it as I build it. This post is long enough already. But now you have the context, you can follow along with the next posts.