Artie | 24 January 2023
Cluster API (CAPI) is one of Kubernetes' sub-projects to bring declarative APIs to cluster creation, configuration, and management. It programmatically configures and deploys K8s clusters on a variety of different infrastructures. You can provision clusters on demand and it integrates well with many tools, and one of them is FluxCD, a continuous delivery tool for Kubernetes.
In the blog post we will cover these main concepts:
We also use this tool in one of our projects for building both long-running production clusters and ephemeral clusters on demand for testing our new software in different environments.
I will show a little demo on how to continuous deliver workload clusters with FluxCD as well, so keep reading!
Cluster API is a multi-cluster management tool for Kubernetes that helps us to create, configure and manage our clusters via a declarative approach through the use of a management cluster that becomes the main cluster from where you deploy workload clusters.
So you just need to install the tools components, in other words make your cluster into a management cluster, and have some yaml
files to deliver a workload cluster in your chosen provider. This becomes even more useful when you use a CD tool that supports the GitOps approach – so you are able to create workload clusters through Git PRs, commits, etc. Workload clusters are simple Kubernetes cluster which is managed by the management one.
So what are the benefits of using Cluster API?
Kubeadm
bootstrap providerCAPI supports many providers for provisioning clusters, beginning with the local solutions such as Docker and ending with the most popular cloud providers (AWS, GCP, Azure). If you are interested in the full list of supported common providers you can find it here.
I would recommend you to use local providers for short-term workload clusters so they can be provisioned within a few minutes without spending extra money, while production variants for long-term clusters can be deployed on cloud platforms.
Basically, we have two general use cases for what we may need the workload clusters for:
Cluster API fully satisfies these needs, because it can provision both long and short running clusters for specific purposes. It can configure or manage workload clusters during the whole life-cycle so you are able to use CAPI as a provisioning tool of your production / staging clusters for long-term projects. Or, if you need to test new software, an ephemeral workload cluster is for you.
Moreover, you can integrate CAPI with Flux CD so you are able to deliver these clusters without accessing the management cluster – just the GitOps way!
The management cluster plays the role of a centralized cluster from where we deliver workload clusters. Basically, it’s just a Kubernetes cluster but with installed CAPI components with these use cases:
Kubeadm
for your control plane and worker nodeskubeconfig
So the main aim of a management cluster is to deliver new workload clusters to our eco-system.
At LiveWyer, we’ve long needed a solution which allows us to just spin up clusters quickly for various uses – whether that be a short term cluster to test something out or a long term cluster for internal use. So we had two main reasons to try CAPI:
From time to time, we have to upgrade our clusters with a new Kubernetes version, so what about upgrading management and workload clusters? It’s okay, no worries, Cluster API supports the upgrading operation.
It’s possible to upgrade the management cluster as well as workload clusters. I won’t cover here how to do it, because it’s a pretty large task, but you can find the high level steps to fully upgrading a cluster here.
Cluster API can be integrated with many tools, but in this blog post I’ll show you how to use CAPI with FluxCD.
It helps us in the automation of many scenarios – for example, you can create ephemeral workload clusters on demand with Flux CD via Git commits.
Basically, Flux integrates with Cluster API so whenever a new cluster is generated Flux has access to it via the kubeconfig
, stored as a secret in your management cluster.
When you have Flux installed on your management cluster you can deploy resources on any workload clusters via the kubeconfig
you specify in the Kustomization files.
kubeConfig:
secretRef:
name: ${CLUSTER_NAME}-kubeconfig # Cluster API creates this for the matching Cluster
Note: Flux Kustomization that will be deployed on the workload cluster should be in the same namespace as a generated cluster.
If the CAPI provider deletes the kubeconfig
when the workload cluster is deleted, then Flux will fail that Kustomization from then because the secret with the kubeconfig
is missing.
Let’s play around with Cluster API in combination with FluxCD. We will have a demo on how to deploy workload clusters with FluxCD.
It’s very convenient to deploy workload clusters with CD tools such as FluxCD or ArgoCD, but I prefer to use FluxCD because of it’s built-in integration with Cluster API.
In order to reproduce this demo you’ll have to:
Note: To create a helm chart with workload cluster you’ll need to generate a template of workload cluster with help of clusterctl generate cluster ...
. CAPI uses different templates for different providers.
I have this setup of terminal:
flux bootstrap
being run for triggering Flux pipelineWith the above, I can produce the demo below:
In the demo, we created a workload cluster with FluxCD, so let’s describe each step in the demo to have a clear picture of what’s going on.
flux bootstrap
to initiate Flux installation as well as the deployment of resources I placed in the specified repo.Also, I think it would be good to take a look at main concepts I used in the demo:
The best practice says “Only store Flux Kustomization in folder which Flux monitors / will monitor”. I agree with that because it helps us to structure all the resources we’d like to deploy. If you wonder what it should look like, or what other ways of structuring there are, then you are welcome to check these examples/docs:
Here I am using the structure of a monorepo model.
.
├── charts
│ └── cluster
│ ├── Chart.yaml
│ ├── templates
│ │ └── ...
│ └── values.yaml
├── clusters
│ └── my-cluster
│ ├── flux-system
│ │ └── ...
│ └── workload-cluster.yaml
└── workload-cluster
├── kustomization.yaml
└── release.yaml
The peak moment in the demo was running flux bootstrap
CLI command so if you want to get to know Flux better, I will recommend you to take a look at this user-guide.
As the last point I showed how I described the Flux Kustomization workload-cluster.yaml
and workload-cluster/
directory.
Pay attention to the spec.path
field, it uses an absolute path of your repository.
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
...
spec:
...
sourceRef:
kind: GitRepository
name: flux-system
path: ./workload-cluster
You can use the same repository to store your helm charts and it’s very convenient, don’t you think?
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
...
spec:
...
chart:
spec:
chart: ./charts/cluster
sourceRef:
kind: GitRepository
name: flux-system
I’ve really enjoyed learning how to use Cluster API, and it’s a really powerful tool for creating k8s clusters via declarative APIs.
What do you think about this tool? What other interesting demos / PoCs you could do with it? Please, let us know in the comment section down below and I would be happy to read your threads.
Do you need help building on-demand Kubernetes clusters? Get in touch and let's work together.
Contact UsAt LiveWyer Labs we innovate through research and development, see what else we've been working on lately.
If you want to stay up to date and be notified when we post new and exciting content, make sure to follow our Linkedin and Medium.