Terraform, Kubernetes, and Argo CD
When I started rebuilding my Kubernetes
cluster, my first thought was to manage
everything using Helm via Terraform. I was
immediately stymied by the fact that the Helm
provider requires the
Kubernetes cluster to be defined in advance: I couldn’t create the cluster in one module, create the
Helm releases in another, and have the input of the latter be the output of the former. In other
words, this pattern wouldn’t work (assuming helm_releases
passes kubeconfig
to the Kubernetes
provider):
Terraformmodule "cluster" {
# [module definition elided]
}
module "helm_releases" {
# [module definition elided]
kubeconfig = module.cluster.outputs.kubeconfig
}
The obvious solution, provided by the nice people on the DSI Discord, is to completely separate creating the cluster from installing applications. In the abstract, the order of operations would be:
Bashpushd cluster && terraform apply && popd
pushd helm_releases && terraform apply && popd
As I was implementing it, though, I came across a persuasive discussion of how Terraform is the wrong layer for managing the applications themselves. That led me to several videos about GitOps (essentially declarative, version-controlled infrastructure) and Argo CD (a declarative continuous delivery tool).[1] I liked the concept and could see that things like Argo Image Updater and Argo Rollouts mostly addressed some well-founded concerns about GitOps I’d encountered before.
That was enough to convince me. My Terraform code need only be responsible for creating the cluster, which I soon had working. One hurdle was the postfix required by webmentiond: the existing apparatus relied on Google, which only allows sending emails from a preconfigured list of IP addresses. However, a Kubernetes cluster consists of multiple nodes that may be replaced at any time. There’s no way to update Google’s list programmatically, and DigitalOcean’s Floating IPs aren’t available for Kubernetes, so I was stuck doing it manually.
- I was sad to find very little non-video content on these subjects.↩