What if you could just pick up a Kubernetes cluster, and move it?
At KubeCon North America 2024 we demo'd just that: live-migrating a Kubernetes cluster (running a stateful Valkey database) from AWS to GCP to Azure in seconds - and with zero downtime or client interruptions.
Felicitas Pojtinger, who's the Head of R&D at Loophole Labs (and our resident KVM whisperer), gave a talk that explored how to build Kubernetes clusters that leverage spot instances and then how to move that cluster in its entirety across cloud providers and even to on-premise infrastructure.
A link to a video of the talk is below and we’ve also posted Felicitas’ full notes from the talk to our github repo.
Architect
In order to migrate complex infrastructure environments like entire Kubernetes clusters, Architect employs a combination of open-source projects that, when used together, allow for seamless, zero-downtime live migrations. Felicitas explains how each project works and how they come together to deliver Architect, the underlying project used in the demo.
At its core, Architect uses:
- PVM - Pagetable-based Virtual Machine (repo): This kernel patch delivers page table virtualization and enables running VMs within VMs without hardware acceleration.
- Firecracker: A hypervisor providing consistent VM behavior across different cloud platforms via CPU templates. Its snapshot and restore features are crucial for portability.
- Silo: An open-source storage engine optimized for tracking changes in VM snapshots, facilitating rapid migration between hosts.
- Drafter: A framework that integrates PVM and Firecracker, offering tools for VM management, networking, health checks, snapshotting, and packaging.
- Conduit: Enables live migration of network connections alongside VM state, ensuring zero downtime during migrations.
──/~\ Architect
──Optimize cluster costs and maximize node utilization, all without modifying your applications or your infrastructure.
Building Reliable Cross-Cloud Kubernetes Clusters on Spot Instances
The talk is available on the CNCF youtube channel:
If you want to know more please join our Discord or follow us on LinkedIn, Twitter, and GitHub.
