Introducing the SQL Server on Kubernetes Operator

Are you considering replatforming your SQL Server workload due to recent vendor changes, but still need high availability and disaster recovery? You’re not alone. One of the challenges with running SQL Server on Kubernetes is that there’s no Kubernetes operator available. That means no automated lifecycle management, no automatic failover, and no standard way to bootstrap an Always On Availability Group on Kubernetes.

I’m excited to share it today as an open-source project: sql-on-k8s-operator. Let’s go.

Why I Built This

That gap…no operator, that means if you want HA and DR for SQL Server on Kubernetes today, you’re assembling it yourself. You need a StatefulSet, a headless service, an HADR endpoint port, a master key and certificate on every replica, those certificates transferred between replicas, CREATE AVAILABILITY GROUP on the primary, ALTER AVAILABILITY GROUP ... JOIN on each secondary, and then some process watching for failures and issuing ALTER AVAILABILITY GROUP ... FAILOVER when the primary goes down. That’s a lot of moving parts, and you have to get the sequencing right every single time. For teams evaluating Kubernetes as a destination for their SQL Server workloads, this is a real blocker. And even when you get all that right, there’s no automatic failover, you have to build that yourself too.

I built the operator to follow the same approach Microsoft’s mssql-server-ha resource agent uses with Pacemaker using sp_server_diagnostics to assess replica health and issuing standard T-SQL DDL for automatic failover.

The result is an operator built with Kubebuilder that manages the full AG lifecycle declaratively. You describe what you want in a custom resource, and the operator keeps the cluster matching that description on every reconcile.

What It Does

The operator introduces two custom resources:

• SQLServerInstance Deploys a standalone SQL Server pod backed by a StatefulSet, a persistent volume, a mssql.conf ConfigMap, and an optional LoadBalancer or ClusterIP service. Great for dev, test, or any workload that doesn’t need AG complexity.

• SQLServerAvailabilityGroup Deploys an N-replica AG with a headless service for stable DNS, HADR endpoints, automatic certificate bootstrapping across replicas, listener service routing, and configurable automatic failover.

Getting Started

You’ll need a running Kubernetes cluster and kubectl. I’m running Docker Desktop with Kubernetes enabled on my Mac. Install the CRDs and deploy the operator in one shot:

kubectl apply -f https://raw.githubusercontent.com/nocentino/sql-on-k8s-operator/main/dist/install.yaml

Confirm it’s up:

kubectl get pods -n sql-on-k8s-operator-system

NAME                                                      READY   STATUS    RESTARTS   AGE
sql-on-k8s-operator-controller-manager-7b8d9f6c4d-xk9vp  2/2     Running   0          30s

Deploying a Standalone Instance

Create the SA password secret, then apply the CR. You can find examples in the config/samples/ directory of the repo.

kubectl create secret generic mssql-secret \
  --from-literal=SA_PASSWORD='YourStrong!Passw0rd'

kubectl apply -f config/samples/sql_v1alpha1_sqlserverinstance.yaml

The operator creates the StatefulSet, ConfigMap, headless service, and an external LoadBalancer service. Once the pod is Ready, connect on port 1433 through the external IP.

Deploying a Three-Replica Availability Group

Create the secret and apply the SQLServerAvailabilityGroup Custom Resource to build an Availability Group, this is TOO easy.

kubectl create secret generic mssql-ag-secret \
  --from-literal=SA_PASSWORD='YourStrong!Passw0rd'

kubectl apply -f config/samples/sql_v1alpha1_sqlserveravailabilitygroup.yaml

The sample SQLServerAvailabilityGroup Custom Resource configures a three-replica AG in EXTERNAL cluster mode, two synchronous replicas for automatic failover, and one asynchronous replica for DR.

Watch the bootstrap happen in the operator logs, below are the last few lines of the logs showing the AG bootstrap is complete:

kubectl logs -n sql-on-k8s-operator-system \
  deployment/sql-on-k8s-operator-controller-manager \
  -c manager -f

...output omitted...
2026-04-12T18:34:13Z	INFO	AG bootstrap complete	{"controller": "sqlserveravailabilitygroup", "controllerGroup": "sql.mssql.microsoft.com", "controllerKind": "SQLServerAvailabilityGroup", "SQLServerAvailabilityGroup": {"name":"mssql-ag","namespace":"default"}, "namespace": "default", "name": "mssql-ag", "reconcileID": "acf05a4c-dcae-46a8-bbee-5f0c69b8beb6", "ag": "AG1"}
2026-04-12T18:34:16Z	INFO	Updated pod AG role label	{"controller": "sqlserveravailabilitygroup", "controllerGroup": "sql.mssql.microsoft.com", "controllerKind": "SQLServerAvailabilityGroup", "SQLServerAvailabilityGroup": {"name":"mssql-ag","namespace":"default"}, "namespace": "default", "name": "mssql-ag", "reconcileID": "acf05a4c-dcae-46a8-bbee-5f0c69b8beb6", "pod": "mssql-ag-0", "role": "primary"}
2026-04-12T18:34:16Z	INFO	Updated pod AG role label	{"controller": "sqlserveravailabilitygroup", "controllerGroup": "sql.mssql.microsoft.com", "controllerKind": "SQLServerAvailabilityGroup", "SQLServerAvailabilityGroup": {"name":"mssql-ag","namespace":"default"}, "namespace": "default", "name": "mssql-ag", "reconcileID": "acf05a4c-dcae-46a8-bbee-5f0c69b8beb6", "pod": "mssql-ag-1", "role": "readable-secondary"}
2026-04-12T18:34:16Z	INFO	Updated pod AG role label	{"controller": "sqlserveravailabilitygroup", "controllerGroup": "sql.mssql.microsoft.com", "controllerKind": "SQLServerAvailabilityGroup", "SQLServerAvailabilityGroup": {"name":"mssql-ag","namespace":"default"}, "namespace": "default", "name": "mssql-ag", "reconcileID": "acf05a4c-dcae-46a8-bbee-5f0c69b8beb6", "pod": "mssql-ag-2", "role": "readable-secondary"}

Bootstrap completes in about 90 seconds on a fresh cluster.

You can check the status of the AG with the command below. The PHASE should be Running and the PRIMARY should be mssql-ag-0.

kubectl get sqlserveravailabilitygroup mssql-ag
NAME       PHASE     PRIMARY      INIT   AGE
mssql-ag   Running   mssql-ag-0   true   50m

You can also get the full YAML output to see the status of the AG and its replicas. Once it’s done, you can check the status of the AG with and confirm the AG is up and running and its replicas are in the correct roles. Looking at the status below, you can see the AG is up, the primary is mssql-ag-0, and the two secondaries are in the readable-secondary role.

kubectl get sqlserveravailabilitygroup mssql-ag -o yaml

apiVersion: sql.mssql.microsoft.com/v1alpha1
kind: SQLServerAvailabilityGroup
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"sql.mssql.microsoft.com/v1alpha1","kind":"SQLServerAvailabilityGroup","metadata":{"annotations":{},"name":"mssql-ag","namespace":"default"},"spec":{"acceptEula":"Y","agName":"AG1","automaticFailover":{"enabled":true,"failoverThresholdSeconds":30,"healthThreshold":"system"},"clusterType":"EXTERNAL","edition":"Developer","endpointPort":5022,"image":"mcr.microsoft.com/mssql/server:2025-latest","listener":{"name":"mssql-ag-listener","port":1433,"serviceType":"LoadBalancer"},"mssqlConf":{"sqlagent.enabled":"true"},"readOnlyListener":{"name":"mssql-ag-listener-ro","port":1433,"serviceType":"LoadBalancer"},"replicas":[{"availabilityMode":"SynchronousCommit","failoverMode":"Automatic","name":"primary"},{"availabilityMode":"SynchronousCommit","failoverMode":"Automatic","name":"secondary-1","readableSecondary":true},{"availabilityMode":"AsynchronousCommit","failoverMode":"Manual","name":"secondary-2","readableSecondary":true}],"resources":{"limits":{"cpu":"2","memory":"4Gi"},"requests":{"cpu":"500m","memory":"2Gi"}},"saPasswordSecretRef":{"key":"SA_PASSWORD","name":"mssql-ag-secret"},"storage":{"dataVolumeSize":"20Gi","reclaimPolicy":"Retain"}}}
  creationTimestamp: "2026-04-12T18:32:59Z"
  generation: 1
  name: mssql-ag
  namespace: default
  resourceVersion: "4007364"
  uid: 6d9ebf4b-3fe8-4504-a5d7-3e4746eca645
spec:
  acceptEula: "Y"
  agName: AG1
  automaticFailover:
    enabled: true
    failoverThresholdSeconds: 30
    healthThreshold: system
  clusterType: EXTERNAL
  edition: Developer
  endpointPort: 5022
  image: mcr.microsoft.com/mssql/server:2025-latest
  listener:
    name: mssql-ag-listener
    port: 1433
    serviceType: LoadBalancer
  mssqlConf:
    sqlagent.enabled: "true"
  readOnlyListener:
    name: mssql-ag-listener-ro
    port: 1433
    serviceType: LoadBalancer
  replicas:
  - availabilityMode: SynchronousCommit
    failoverMode: Automatic
    name: primary
  - availabilityMode: SynchronousCommit
    failoverMode: Automatic
    name: secondary-1
    readableSecondary: true
  - availabilityMode: AsynchronousCommit
    failoverMode: Manual
    name: secondary-2
    readableSecondary: true
  resources:
    limits:
      cpu: "2"
      memory: 4Gi
    requests:
      cpu: 500m
      memory: 2Gi
  saPasswordSecretRef:
    key: SA_PASSWORD
    name: mssql-ag-secret
  storage:
    accessModes:
    - ReadWriteOnce
    dataVolumeSize: 20Gi
    reclaimPolicy: Retain
status:
  initializationComplete: true
  phase: Running
  primaryReplica: mssql-ag-0
  replicaStatuses:
  - connected: true
    lastSeenPrimary: "2026-04-12T18:34:24Z"
    name: mssql-ag-0
    role: PRIMARY
    synchronizationState: SYNCHRONIZED
  - connected: true
    name: mssql-ag-1
    role: SECONDARY
    synchronizationState: SYNCHRONIZED
  - connected: true
    name: mssql-ag-2
    role: SECONDARY
    synchronizationState: SYNCHRONIZING

Adding a Database to the Availability Group

But what’s an AG without a database. So let’s add one. The command below creates the database if it doesn’t already exist, takes a full backup to establish the log chain, and then explicitly adds it to the AG. The operator uses direct seeding to get the database to the other replicas in the AG.

kubectl exec -it mssql-ag-0 -- /opt/mssql-tools18/bin/sqlcmd \
  -S localhost -U sa -P 'YourStrong!Passw0rd' -No \
  -Q "
IF DB_ID('TestDB') IS NULL CREATE DATABASE TestDB;
BACKUP DATABASE TestDB
    TO DISK = '/var/opt/mssql/data/TestDB.bak'
    WITH INIT, FORMAT;
GO
ALTER AVAILABILITY GROUP [AG1] ADD DATABASE TestDB;
GO
"

Query the AG replica state to verify the database replication is healthy:

kubectl exec -it mssql-ag-0 -- /opt/mssql-tools18/bin/sqlcmd \
  -S localhost -U sa -P 'YourStrong!Passw0rd' -No \
  -Q "
SELECT
    ar.replica_server_name,
    db.name AS database_name,
    drs.synchronization_state_desc,
    drs.synchronization_health_desc
FROM sys.dm_hadr_database_replica_states drs
JOIN sys.dm_hadr_availability_replica_states ars
    ON drs.replica_id = ars.replica_id
JOIN sys.availability_replicas ar
    ON drs.replica_id = ar.replica_id
JOIN sys.databases db
    ON drs.database_id = db.database_id
"

You should see SYNCHRONIZED and HEALTHY for the synchronous replicas, and SYNCHRONIZING and HEALTHY for the asynchronous replica:

How the Bootstrap Works

The operator runs these steps:

1. Waits for all replica pods to be Ready, there is a readiness probe on the SQL Server container so we know it’s up and accepting connections.
2. Creates a master key, a self-signed certificate, and an HADR endpoint on each replica
3. Transfers certificates between replicas
4. Installs the replicas certificates and creates the logins needed for endpoint authentication
5. Verifies HADR endpoint TCP reachability between every pair of replicas before proceeding
6. Runs CREATE AVAILABILITY GROUP with SEEDING_MODE = AUTOMATIC on the primary no backup/restore required for secondaries to synchronize
7. Issues ALTER AVAILABILITY GROUP ... JOIN on each secondary, then grants CREATE ANY DATABASE for automatic seeding
8. Labels each pod with sql.mssql.microsoft.com/ag-role=primary or readable-secondary so listener service selectors always route to the right pod

Automatic Failover

With clusterType: EXTERNAL and automaticFailover.enabled: true, the operator acts as the external cluster manager. If the primary pod becomes NotReady and stays that way past failoverThresholdSeconds, or sp_server_diagnostics reports a health issue, the operator selects the best synchronized secondary and issues a FAILOVER command. But for this blog post I’m going to manually failover the AG. When using the EXTERNAL cluster type, you must set the external_cluster session context before running the failover command:

kubectl exec -it mssql-ag-1 -- /opt/mssql-tools18/bin/sqlcmd \
  -S localhost -U sa -P 'YourStrong!Passw0rd' -No \
  -Q "
EXEC sp_set_session_context @key = N'external_cluster', @value = N'yes';
ALTER AVAILABILITY GROUP AG1 FAILOVER;
"

After the failover, query the replica state to verify the new primary and synchronization status. You’ll see SYNCHRONIZED and HEALTHY for the new primary mssql-ag-1, and SYNCHRONIZING and HEALTHY for the asynchronous replica, mssql-ag-2. The asynchronous replica will never reach SYNCHRONIZED because it’s in AsynchronousCommit mode. It may be NOT SYNCHRONIZING for a short time once it reconnects it will change to SYNCHRONIZING and HEALTHY again.

kubectl exec mssql-ag-1 -- \
  /opt/mssql-tools18/bin/sqlcmd \
  -S localhost,1433 -U sa -P 'YourStrong!Passw0rd' -No -C \
  -Q "
SET NOCOUNT ON;
SELECT
    r.replica_server_name  AS Pod,
    rs.role_desc           AS Role,
    rs.synchronization_health_desc AS [Sync Health],
    ISNULL(drs.synchronization_state_desc, '—') AS [Sync State]
FROM sys.dm_hadr_availability_replica_states rs
JOIN sys.availability_replicas r
  ON rs.replica_id = r.replica_id
LEFT JOIN sys.dm_hadr_database_replica_states drs
  ON rs.replica_id = drs.replica_id
ORDER BY rs.role_desc DESC, r.replica_server_name;
"

The operator also handles headless AGs where a primary crash leaves the AG in a RESOLVING state with no elected primary. The operator detects that immediately and promotes the best available secondary without waiting out the full threshold timer. That’s the fast path, and it’s the path you want in a genuine crash scenario.

After any failover, pod labels and the listener service selector are updated automatically. Client connections route to the new primary without any manual intervention.

Testing the Full Lifecycle

Claude and I built a test script test-ag-failover.sh that automates the complete AG lifecycle to verify everything works end-to-end. This will deploy the operator, create the AG, add a database, failover both planned and unplanned, and clean up.

Run all five phases with a single command:

./test-ag-failover.sh all

At every state transition, the script captures a two-layer health snapshot: Kubernetes pod readiness and SQL Server replica state from sys.dm_hadr_availability_replica_states. Here’s what the output looks like after a planned failover:

Notice that mssql-ag-2 shows SYNCHRONIZING rather than SYNCHRONIZED that’s expected and correct. It’s the asynchronous replica in AsynchronousCommit mode, and SYNCHRONIZING is its healthy steady state.

Key Features at a Glance

• Automated AG bootstrap Certificate creation, HADR endpoints, CREATE AVAILABILITY GROUP, and JOIN handled by the operator, no manual T-SQL required
• Automatic seeding Replicas synchronize via SEEDING_MODE = AUTOMATIC, no backup/restore required
• Automatic unplanned failover Configurable threshold timer with a fast headless-AG detection path for crash scenarios
• Planned failover The operator supports a graceful primary switch at any time
• PodAntiAffinity Replica pods are preferred-scheduled across different Kubernetes nodes
• PV reclaim policy Bound PersistentVolumes are patched to Retain by default so data files survive CR deletion
• Dynamic pod labeling AG role labels (primary, readable-secondary) drive listener and read-only service routing automatically
• mssql.conf support Arbitrary SQL Server config options passed via CR fields, no custom image required
• End-to-end test script Five-phase lifecycle test with two-layer health checks at every state transition

Get the Code

The project is at github.com/nocentino/sql-on-k8s-operator. The config/samples/ directory has ready-to-use CRs for both resource types, and test-ag-failover.sh runs the full scenario against a live cluster. Clone the repo, deploy it against your cluster, and let me know how it works in your environment.

Disclaimer

This project is open source and is not supported by Microsoft or any other vendor. There is no SLA, no enterprise support contract, and no guarantees. It works well in my lab, but you should evaluate it carefully before running it anywhere near production. I’m sharing it because I think the community needs it and I want to build momentum behind it as a serious open-source effort. If it solves a problem you have, I’d love to hear about it. If you find bugs, open an issue. If you want to contribute, pull requests are very welcome.

Wrapping Up

The HADR story for SQL Server on Kubernetes is finally here. This operator handles the hard parts bootstrap, certificate exchange, failover detection, listener routing and keeps reconciling so drift doesn’t build up over time. It’s working well in my lab across both planned and unplanned failover scenarios, and I’m excited to get it in front of more environments. Clone the repo and give it a try.