GitOps handles the parts of a deployment that have a clean before-and-after: a Deployment’s pod template changes, the controller reconciles, done. A schema migration does not fit that shape. It has an order dependency the rest of the pipeline does not naturally enforce (the migration must run before the new code that expects it), and it is frequently not reversible in the way a code rollback is. The site’s own GitOps reference names this as one of the model’s real blind spots: GitOps describes a target state well, a process with a before and after badly, and schema migrations are exactly that kind of process.
Ordering: the migration runs before the rollout, not during it
The mechanical problem is simple to state: application code that queries a new column must not start running before that column exists. Two common ways to enforce the order in a pipeline.
A dedicated CI job, gated to run before the deploy step:
# GitHub Actions, simplified
jobs:
migrate:
steps:
- run: migrate -database "$DATABASE_URL" -path db/migrations up
deploy:
needs: migrate # deploy cannot start until migrate succeeds
steps:
- run: kubectl set image deployment/app app=registry/app:${{ github.sha }}
Or, inside Kubernetes, a Job run as a Helm pre-install/pre-upgrade hook, which the chart’s own lifecycle blocks on:
apiVersion: batch/v1
kind: Job
metadata:
name: db-migrate
annotations:
"helm.sh/hook": pre-upgrade
"helm.sh/hook-weight": "-1"
"helm.sh/hook-delete-policy": before-hook-creation
spec:
template:
spec:
containers:
- name: migrate
image: migrate/migrate
args: ["-path", "/migrations", "-database", "$(DATABASE_URL)", "up"]
restartPolicy: Never
Either shape gives you the same guarantee: the rollout does not proceed while the migration job is still running or has failed. What it does not give you, on its own, is safety for the window where old and new code both run against the new schema at once, which is the normal state during any rolling deployment.
Expand/contract: the pattern that survives two versions running at once
During a rollout, old and new pods serve traffic against the same database simultaneously, sometimes for minutes. A migration that changes a column’s meaning in one step breaks whichever version is not expecting it. Expand/contract splits the change into steps that are each individually compatible with both versions.
Renaming a column, done safely, looks like three separate migrations across three separate deploys, not one:
-- Migration 1 (expand): add the new column, keep the old one, backfill
ALTER TABLE users ADD COLUMN email_address text;
UPDATE users SET email_address = email WHERE email_address IS NULL;
-- App deploy 1: write to BOTH columns, read from the OLD one.
-- Old pods (pre-migration code) are unaffected; they never see the new column.
-- App deploy 2: write to BOTH columns still, read from the NEW one.
-- Only ships once deploy 1 has been running long enough that every
-- row written since has both columns populated.
-- Migration 2 (contract): drop the old column, once nothing reads it
ALTER TABLE users DROP COLUMN email;
Each step alone is a no-op for whichever code version does not know about it yet. The discipline this demands is patience: contract only happens once you are certain no running version still depends on the old shape, which in practice means at least one full deploy cycle after the expand, sometimes more if rollback of the previous release is still on the table.
The rollback myth
A code rollback is cheap: revert the commit, redeploy the old image, the previous binary runs against whatever it always expected. A migration rollback is not the same operation wearing a different name, and treating it as such is where teams get hurt.
An additive migration (add a column, add a table) rolls back cleanly: drop what you added, nothing else was touched. A destructive migration does not: DROP COLUMN deletes the data in it, and no “down migration” recreates data that no longer exists anywhere. ALTER COLUMN price TYPE integer truncates every fractional value in that column; the down migration can restore the type, not the digits it discarded. The honest rule is that a migration is only as reversible as the information it destroys, which is precisely why expand/contract exists: additive-only steps stay reversible for as long as they need to, and the one genuinely destructive step (the contract) is deferred until you no longer need the rollback option at all.
What to check before the next migration
Before merging a migration, ask two questions the pipeline itself cannot ask for you. Does this migration remain valid if the previous application version is still running against it for the next several minutes? And if this needs to be undone in five minutes, does the down migration restore behavior, or does it only restore a schema shape with the data already gone? A migration that fails either question is not ready to ship as a single step; split it. This kind of sequencing discipline is one of the pieces I bring into an industrializing CI/CD engagement: a pipeline that deploys code safely but ships schema changes as an afterthought still has an outage waiting in it.