Part 3 - Kubernetes Multi-Clusters
Summary
In this Part 3, we’ll explore several multi-cluster architectures, namely:
- Sinlgle cluster
- Replicated multi-cluster
- Kubernetes Centric
- Network Centric
- GitOps cluster fleet
- Sinlgle cluster
We provide an overview of various methods for integrating clusters, including multi-cluster management, application deployments, and Control Plane synchronization.
If you’re looking to operate ‘Kubernetes at scale‘, you don’t want to miss this post!
Lastly, we’ll present my conclusions to this three-part series.
Recap
In Part 2, we compared single vs multi-cluster architectures and detailed several top-level benefits multi-cluster Kubernetes has to offer, particularly for high-performance and mission-critical workloads.
We took a look at what the cloud service providers are currently offering in the multi-cluster space and what a DIY Multi-Cluster can mean for you and your business.
What is a Multi-Cluster Architecture?
Refresher
A multi-cluster architecture enables the distribution of applications across independent, geographically dispersed clusters.
Cluster Architectures
Single cluster
Kubernetes does not provide multi-cluster functions out-of-the-box.
The Single Cluster approach is the simplest and most widely adopted architecture today however, it is technically, not a multi-cluster solution and is listed here for reference.
The architecture provides high-availability for applications, service recovery, self-healing and performance throughput perfectly suited to a number of workloads.
It is not however, recommended for mission critical applications (see Part-1) and does not easily support cross-region deployments. Today, this is the most commonly used pattern, with most organizations contending with sprawls of hundreds of Kubernetes Single clusters.
Replicated
A replicated cluster architecture is in effect, a multi-cluster deployment.
Note: this is not about having multiple clusters (which are common), but involves effectively controlling multiple clusters as one unified platform and coordinating application deployments across them.
In a replicated architecture, each cluster is a copy or replica – of one base cluster. All clusters are identical.
Applications are deployed across multiple clusters, each having their own Control Plane. In effect, this is a distributed and independent architecture where every cluster has the necessary resources to offer the service in complete isolation from every other cluster.
This pattern provides a great level of flexibility and scale.
A multi-cluster deployment is not just about having multiple clusters, it involves effectively coordinating and controlling them as a unified platform.
Coordinating Pod deployments across multiple clusters is much more complex then with single clusters. For starters, you’ll need some sort of global load-balancer in addition to each cluster’s local load-balancer if you are to direct user traffic to the closest cluster to their location.
It’s important to note that Kubernetes does not provide multi-cluster functions out-of-the-box!
If you want to reduce management overhead and avoid duplicating efforts, it’s vital to centralize management with this architecture.
The tooling you select will be critical to the success of your multi-cluster strategy.
Replication Approaches
Lets take a brief and relatively simply look at the complex world of multi-cluster management. We’ll discuss the top two replicated approaches.
Kubernetes Centric
Also referred to as the “Control-Plane centric” approach, this method looks to synchronise control plane objects, application settings and deployment configurations to each cluster that will run an application across multiple clusters.
Figure 4. Example: Control Plane Centric Multi-Cluster (Cilium Cluster Mesh)
Source: Blog by cilium.io
Figure 4 illustrates an example of how Cilium Cluster Mesh performs cross-cluster, Kubernetes control plane synchronisation using a complex combination of cluster agents and etcd proxy servers, over a TLS encrypted channel between clusters.
Network Centric
This type of approach typically uses a Service Mesh and enables pods on multiple clusters to communicate directly with one another. It relies on enabling network communication between different clusters to implement resource replication and keep clusters in synchronous with one another.
Most notably, this type of multi-cluster facilitates cross-region topologies allowing for global application deployments.
When using a network-centric approach, cluster nodes must be capable of routing Pod IP CIDRs between clusters, or alternatively, nodes need to forego using an IPv4 overlay network altogether and implement IPv6 addressing.
Figure 5. Example: Service Mesh, Network Centric Multi-Cluster
Source: Blog by istio.io
Hot Tip
A Service Mesh inherently, increases latency and complexity. Using a highly efficient Kernel based, eBPF data-path networking structure such as Cilium’s Cluster Mesh can potentially reduce latency by up to 70%.
GitOps Cluster Fleet
The greater the complexity, the greater the scope for failure.
The greater the complexity, the greater the scope for failure—so here, we dispense entirely with complex replication mechanisms.
GitOps acts as a single source of truth for your cluster fleet.
A centralized and version controlled Git repository holds both cluster and application configurations. A GitOps controller deployed on each cluster in your fleet, periodically reads the desired state config from these Git repositories and instructs it’s Control Plane API server to implement the desired state.
This approach, centralizes fleet management and facilitates easy rollback to previous states.
All configurations are made using just one language, the Kubernetes API which uses declarative yaml syntax to describe and configure Kubernetes objects. This standard gives developers and operations staff alike a common, unified language to manage workloads and their platforms.
Figure 6. – GitOps Fleet Management
Cluster Management Tools
By using appropriate tooling and establishing robust processes, it is possible to significantly reduce the load on administrative systems with this architecture and easily scale to hundreds of clusters with minimal operational complexity.
The tooling you select will be critical to the success of your multi-cluster architecture.
Advanced application rollout strategies, such as blue/green and canary deployments, can be accomplished by installing appropriate deployment controllers onto your clusters.
Use appropriate tools to help you automate and streamline application upgrades while ensuring service availability, even during periods of continued high volume traffic.
Additionally, the right tools can greatly ease the administrative burden of coordinating application rollouts to large numbers of clusters simultaneously. They can help ensure each cluster is properly configured and up-to-date, which is essential for maintaining a secure and stable environment.
Conclusion
We’ve provided a summary of Kubernetes multi-cluster architectures and emphasized the significance of choosing the right tooling for managing multiple clusters.
Three main architectures were discussed:
- Single cluster
- Replicated multi-cluster
- GitOps fleet clusters
Furthermore, we’ve explored two approaches to replicated multi-clustering:
- Kubernetes Centric
- Network Centric
To fully leverage the advantages of multi-cluster Kubernetes, it is important for your teams to deploy and manage these environments without being overwhelmed by complex management tasks.
Effectively deploying and managing multi-cluster Kubernetes can ultimately give enterprises a significant competitive advantage; offering greater geographical reach, resilience, flexibility and data sovereignty compliance.
Key points
Approximate reading time
8 min
Share posts
Other posts
Tony Barganski is a Principal Consultant at a tech consultancy in London, where he focuses on meeting the cloud computing needs of London’s financial institutions. Tony consistently delivers impactful solutions and has played a pivotal role in driving business advancements for his clients.
2 Responses