Terraform, Kubernetes, and Argo CD

When I started rebuilding my Kubernetes cluster, my first thought was to manage everything using Helm via Terraform. I was immediately stymied by the fact that the Helm provider requires the Kubernetes cluster to be defined in advance: I couldn’t create the cluster in one module, create the Helm releases in another, and have the input of the latter be the output of the former. In other words, this pattern wouldn’t work (assuming helm_releases passes kubeconfig to the Kubernetes provider):

module "cluster" {
  # [module definition elided]
}

module "helm_releases" {
  # [module definition elided]
  
  kubeconfig = module.cluster.outputs.kubeconfig
}

The obvious solution, provided by the nice people on the DSI Discord, is to completely separate creating the cluster from installing applications. In the abstract, the order of operations would be:

pushd cluster && terraform apply && popd
pushd helm_releases && terraform apply && popd

As I was implementing it, though, I came across a persuasive discussion of how Terraform is the wrong layer for managing the applications themselves. That led me to several videos about GitOps (essentially declarative, version-controlled infrastructure) and Argo CD (a declarative continuous delivery tool).^[1] I liked the concept and could see that things like Argo Image Updater and Argo Rollouts mostly addressed some well-founded concerns about GitOps I’d encountered before.

That was enough to convince me. My Terraform code need only be responsible for creating the cluster, which I soon had working. One hurdle was the postfix required by webmentiond: the existing apparatus relied on Google, which only allows sending emails from a preconfigured list of IP addresses. However, a Kubernetes cluster consists of multiple nodes that may be replaced at any time. There’s no way to update Google’s list programmatically, and DigitalOcean’s Floating IPs aren’t available for Kubernetes, so I was stuck doing it manually.

Next in series: The Ingress Controller: Contour(#3 in The Death and Rebirth of a Cluster)