Kubernetes

In my planned failover walkthrough, I showed what happens when you deliberately move the primary role to another replica. That’s the easy case. Now I want to show what happens when the primary pod just disappears unexpectedly, like during a node failure or a container crash. No graceful shutdown, no demotion, just gone.

I ran two test scenarios, each cycling the primary role across all three pods by force-deleting the current primary three times in a row. First, a 5GB TPC-C database idle. Then, that same 5GB database under sustained HammerDB TPC-C load. Six force-deletes total, six successful automatic failovers. I’ll walk through the error log from the promoted replica, the operator’s detection and recovery behavior, and the full timing data.

When building the sql-on-k8s-operator, I wanted to make sure it could handle both planned and unplanned failovers. The easy case is a planned failover, where you deliberately move the primary role to another replica. The harder case is an unplanned failover, where the primary pod just disappears. The operator needs to handle both.

I recently ran a full planned failover rotation on a three-replica SQL Server Availability Group managed by sql-on-k8s-operator, and I want to show you exactly what happens inside SQL Server and the operator during each hop. If you’ve been following my Introducing the SQL Server on Kubernetes Operator post, this is the logical next step: what does the error log actually look like during a planned failover, what does the operator do in response, and how long does the whole thing take?

Are you considering replatforming your SQL Server workload due to recent vendor changes, but still need high availability and disaster recovery? You’re not alone. One of the challenges with running SQL Server on Kubernetes is that there’s no Kubernetes operator available. That means no automated lifecycle management, no automatic failover, and no standard way to bootstrap an Always On Availability Group on Kubernetes.

I’m excited to share it today as an open-source project: sql-on-k8s-operator. Let’s go.

Update (February 2026): This post has been updated to reflect the current best practices for monitoring FlashArray using OpenMetrics. Starting with Purity//FA 6.7.x, FlashArray includes a native OpenMetrics exporter, eliminating the need for a separate exporter service/container. The configuration examples below now demonstrate direct-to-array scraping.

This post introduces you to monitoring your Pure Storage FlashArray using OpenMetrics. It shows you how to get started quickly using Docker Compose to monitor your Pure Storage FlashArray environment.

We’re working through the major refresh of my Certified Kubernetes Administrator series at Pluralsight!

The next course “Certified Kubernetes Administrator: Performing Cluster Version Upgrades” in the updated series is now available on Pluralsight here! If you want to learn about the course, check out the trailer here, or if you’re going to dive right in, check it out here! This course will teach you the how to perform worker node maintenance and upgrading a Kubernetes cluster using kubeadm.

This post shows you how to install containerd as the container runtime in a Kubernetes cluster. I will also cover setting the cgroup driver for containerd to systemd, which is the preferred cgroup driver for Kubernetes.

In Kubernetes version 1.20 Docker was deprecated as a container runtime in a Kubernetes cluster and support was removed in 1.22. Kubernetes 1.26 requires that you use a runtime that conforms with the Container Runtime Interface (CRI). containerd is a CRI-compatible container runtime and is one of the supported options you have as a container runtime in this post-Docker/Kubernetes world. To be clear, you use container images created with Docker in containerd. containerd will start and run the container in your Kubernetes cluster. This post was previously published in February 2021. This is an updated version with the latest installation and configuration steps.

We’re working through the major refresh of my Certified Kubernetes Administrator series at Pluralsight!

The next course “Certified Kubernetes Administrator: Working With Your Cluster” in the updated series is now available on Pluralsight here! If you want to learn about the course, check out the trailer here, or if you’re going to dive right in, check it out here! This course will teach you the first steps in interacting with a Kubernetes cluster using kubectl.

We’re kicking off a major refresh of my Certified Kubernetes Administrator series at Pluralsight!

The second course “Certified Kubernetes Administrator: Using kubeadm to Install a Basic Cluster” in the updated series is now available on Pluralsight here! If you want to learn about the course, check out the trailer here, or if you’re going to dive right in, check it out here! This course will teach you the fundamentals needed to get using kubeadm to install a basic Kubernetes cluster!

We’re kicking off a major refresh of my Certified Kubernetes Administrator series at Pluralsight!

The first course “Certified Kubernetes Administrator: Kubernetes Foundations” in the updated series is now available on Pluralsight here! If you want to learn about the course, check out the trailer here, or if you’re going to dive right in, check it out here! This course will teach you the fundamentals needed to get started with Kubernetes!

My new course “Securing Azure Kubernetes Service (AKS) Clusters”, co-authored with my good friend and colleague Ben E. Weissman, is now available on Pluralsight here! If you want to learn about the course, check out the trailer here, or if you’re going to dive right in, check it out here!

This course completes the learning path, Managing and Orchestrating Containers with Azure Kubernetes Service (AKS), that Ben and I built together.